Services

Common problems and solutions if you are experiencing issues when writing, building or running services on the Rover.

Uploading my service fails with no valid service.yaml file found in this directory

Cause

The service directory that you are trying to upload does not contain a valid service.yaml file at its root.

Sanity Checks

• Did you specify the correct path to upload?
• Did you put a service.yaml file at the root of your service directory?
• Does your service.yaml file contain all the required fields?
• Is your service.yaml file valid YAML?
  • Check indentation: do you use tabs and spaces interchangeably?
  • Check unclosed arrays

My service fails to build

Cause

The build command specified in your service.yaml file cannot be executed by roverd, or no build command is specified.

Sanity Checks

• Did you upload the latest version of your service?
  • Try altering the version field in your service.yaml file and uploading it. Does it show up in roverctl?
• Is the build command a valid bash command?
• Is the build command a valid executable, relative to the root directory of your service?
• Is the build command valid when executed as the debix user?

Trouble Shooting

Specify a build command that always works

If your service does not rely on "building" anything. You can specify a build command that always works, such as echo or exit 0. For example:

service.yaml

...
commands:
  build: echo built # will echo and exit safely
...

Build the service without roverd

If your service needs a build step, for instance to compile or transpile code, try running the build command on the Rover. Connect to the Rover over SSH or UART and cd into the service directory to run the build command as the debix user. Use the build output to confirm that building works.

If building works, this is most likely a roverd issue. Reach out to the ASE team.

Ensure that all dependencies are installed on the Rover

If your service relies on global dependencies (such as apt packages, pip packages, shared objects, global header files or libraries), make sure that these are also installed on the Rover for the debix user. Read about how roverd executes your build command here.

My service crashes with failed to create read stream

Cause

Your service code tries to create a read stream that is not defined as input in your service.yaml definition.

Trouble Shooting

Confirm that the input streams are defined

Confirm that the inputs in your service.yaml and the code in your service correspond. For example, if you create a read stream like this:

...
readStream := service.GetReadStream("imaging", "path")
...

Your service.yaml definition should contain the following declaration:

...
inputs:
  - service: imaging
    streams:
      - path
...

Confirm that the bootspec is injected properly by roverd

Check which bootspec is injected by printing the ASE_SERVICE environment variable. You can do this in your service code, or by replacing the run command in your service.yaml file.

...
commands:
  run: echo $ASE_SERVICE
...

Then, check the logs through roverctl-web, or by running the roverctl logs command. If the stream is not defined, this is a roverd issue.

My service crashes with failed to create write stream

Cause

Your service code tries to create a write stream that is not defined as output in your service.yaml definition.

Trouble Shooting

Confirm that the output streams are defined

Confirm that the outputs in your service.yaml and the code in your service correspond. For example, if you create a write stream like this:

...
writeStream := service.GetWriteStream("decision")
...

Your service.yaml definition should contain the following declaration:

...
outputs:
  - decision
...

Confirm that the bootspec is injected properly by roverd

Check which bootspec is injected by printing the ASE_SERVICE environment variable. You can do this in your service code, or by replacing the run command in your service.yaml file.

...
commands:
  run: echo $ASE_SERVICE
...

Then, check the logs through roverctl-web, or by running the roverctl logs command. If the stream is not defined, this is a roverd issue.

My service crashes with failed to get configuration value

Cause

Your service code tries to access a configuration value that is not defined in your service.yaml definition.

Trouble Shooting

Confirm that the configuration value is defined

Confirm that the configuration values and types in your service.yaml and the code in your service correspond. For example, if you access a value like this:

...
configuration.GetFloatSafe("speed")
...

Your service.yaml definition should contain the following declaration:

...
configuration:
  - name: speed
    value: 1.0
    type: number 
...

Confirm that the bootspec is injected properly by roverd

Check which bootspec is injected by printing the ASE_SERVICE environment variable. You can do this in your service code, or by replacing the run command in your service.yaml file.

...
commands:
  run: echo $ASE_SERVICE
...

Then, check the logs through roverctl-web, or by running the roverctl logs command. If the stream is not defined, this is a roverd issue.

My service crashes with an exit code

Cause

Your service exits unexpectedly.

Sanity Checks

• Did you upload the latest version of your service?
  • Try altering the version field in your service.yaml file and uploading it. Does it show up in roverctl?
• Is the run command a valid bash command?
• Is the run command a valid executable, relative to the root directory of your service?
• Is the run command valid when executed as the debix user?

Trouble Shooting

Read the logs

You can view the logs through roverctl-web, or by running the roverctl logs command. This should be your first guide.

Try to run your service without roverd

You can read how to do so here.

Try to run your service locally, without roverd

You can read how to do so here.

My service's logs are not captured

Cause

The buffers for STDOUT and/or STDERR are not flushed, so roverd cannot capture output from them.

Trouble Shooting

In your service's source code, make sure to flush STDOUT and STDERR.

My service slows down when reading from other services

Cause

The roverlib calls to read from and write to streams are blocking and your service does not accommodate for this.

Trouble Shooting

If you don't want to block your main loop, use a threaded solution that is native to your service's programming language to read and write from streams. Read more in this tutorial.

The protobuf messages that I write are not plotted in the roverctl-web debug view

Sanity Checks

• Did you start roverctl-web in debug mode?
• Did you enable the "debug mode" toggle?
• Did your service start correctly? (Check its logs)
• Are you using Write() on your write stream?
  • Debugging does not work with WriteBytes()

Cause

Most likely, you forgot to specify a SensorId, Timestamp and Status field. If these fields are omitted, roverctl-web will not plot your data. Note that only numeric values and camera data (JPEG bytes) can be plotted.

Trouble Shooting

Make sure the messages you send follow a structure that is identical to the one described in this guide.