Refresh Rates and Blocking Calls

When developing a service that depends on multiple services, you might experience unexpected slow-downs or poor performance. In this tutorial we will explain why this happens and how you can resolve this.

Elias Groot

Founding Member, Ex-Software Lead and Ex-Project Administrator

Each roverlib uses the abstraction of streams to arrange communication (writing and reading) between services. The underlying implementation uses blocking calls for this. That means that a call will only return when data is available to be read. Consider the example of a Go service below.

main.go

...
// Initialize read stream
rs := service.GetReadStream("imaging", "path")

// Main loop
for {
    // Try to read from imaging
    data, err := rs.Read()
    // This point is only reached when there is data to read!
    // otherwise, it will block until there is data available

    // Do something with this data
    // ...
}
...

Now consider that imaging only processes one image per second. It then writes data at 1 Hz. That means that, in this blocking setup, the main loop will be executed at 1 Hz as well.

Unblocking the Main Loop using Threads

In many cases, blocking the main loop like above is fine. We do not want to process anything if there is no imaging data ready anyway. However, imagine the scenario where you have a fast service (a distance sensor) that publishes at 100 Hz and a slow service (the imaging service) at 1 Hz. You read both in the same loop:

main.go

...
// Initialize read stream from fast service
fast := service.GetReadStream("distance", "distance")
// Initialize read stream from slow service
slow := service.GetReadStream("imaging", "path")

// Main loop
for {
    // Try to read from distance
    data, err := fast.Read()
    // This point is only reached when there is data to read!
    // otherwise, it will block until there is data available

    // Brake when we are close to hitting a wall
    if data.GetDistanceOutput().Distance < 0.5 {
        // braking logic here ...
    }

    // Try to read from imaging
    data, err = slow.Read()
    // This point is only reached when there is data to read!
    // otherwise, it will block until there is data available

    // Do something with this data
    // ...
}
...

On the first iteration, this is still safe: if the distance sensor detects a wall nearby, it will brake the Rover. However, when the second iteration is reached (after reading from imaging), the distance sensor has already written messages, which - due to guaranteed delivery - will be processed with a large latency now. The loop executes at 1 Hz, while the distance sensor publishes at 100 Hz. Hence, the queue of remaining distance sensor messages builds up quickly.

In general, when reading from multiple services in a loop like this, the loop will be executed at the speed of the slowest service. Again, this is not a problem per se, but when you want to perform other tasks as well, this might yield unexpected results. The solution is to use threading in your language of choice.

In our Go example, we can use goroutines: a light-weight alternative to using system threads. In this case, we would split up the loops into two: one performed in the main thread, and one in parallel in a goroutine.

main.go

...
// Initialize read stream from fast service
fast := service.GetReadStream("distance", "distance")
// Initialize read stream from slow service
slow := service.GetReadStream("imaging", "path")

// Fast loop, executed in a goroutine
go func() {
    for {
        // Try to read from distance
        data, err := fast.Read()
        // This point is only reached when there is data to read!
        // otherwise, it will block until there is data available

        // Brake when we are close to hitting a wall
        if data.GetDistanceOutput().Distance < 0.5 {
            // braking logic here ...
        }
    }
}()

// Main loop
for {
    // Try to read from imaging
    data, err = slow.Read()
    // This point is only reached when there is data to read!
    // otherwise, it will block until there is data available

    // Do something with this data
    // ...
}
...

Unblocking the Main Loop using DONTWAIT

It is important to understand that both .Read() and .Write() (and their underlying .ReadBytes() and .WriteBytes()) calls are blocking in all roverlibs. You can find the underlying implementation here. We recommend using a threaded solution (like goroutines) in your language of choice if you do not want to block your main loop on reading or writing.

In case you really do not want to set up threading, you can use custom ZMQ sockets yourself, and use the DONTWAIT flag. We use this in the official ASE transceiver service, and you can find the relevant example code here.