0
$\begingroup$

Rosanswers logo

I'm wondering if I'm setting up different timers, do they run in different threads using roscpp? The background is that one Timer callback is taking a longer time to be executed and, therefore, I want to run it at a lower frequency. But it should not interrupt the execution frequency of ros::spin() (because of the callbacks there) and the other Timer callback.

So my open questions are:

  • Do different Timer callbacks run in different threads?
  • Do callback functions of the same Timer run in different threads or what is happening if the frequency of a Timer is too high and execution of callback takes longer than this?
  • Do Timers run in a different thread than ros::spin(), or is ros::spin() setting the minimal time? From what I've read, this is the case and the Timer is executed during ros::spin(). If callbacks during ros::spin() take longer than the Timer duration, what will happen?
  • And as a possible solution: If I switch to AsyncSpinner, does this also affect the Timer callbacks?

What I'm doing now is described in this solution: https://answers.ros.org/question/53055/ros-callbacks-threads-and-spinning/?answer=53088#post-id-53088

Typically, if you want to do time-consuming computations such as "leg detection", the callbacks are definitively not the place to do it.

Just, copy your data to a LegDetector object instance and call in your main thread the method that will do the heavy work.

However, how should this not block my other callbacks? I'm calling the computational intense function during while(ros::ok()) and it is blocking the ROS spinner and, therefore, every other callback. So what is the intention behind this solution?

Sources: http://wiki.ros.org/roscpp/Overview/Timers https://roboticsbackend.com/roscpp-timer-with-ros-publish-data-at-a-fixed-rate/ :

Once created, the timer will return so the following of the code will be executed. It will then, in another thread, call the callback every X seconds, where X is the first parameter you gave to the Timer.

https://answers.ros.org/question/240388/do-topic-callbacks-and-timer-callbacks-run-in-the-same-thread/ :

No, topicCallback and timerCallback cannot be executed in parallel, as long as you are using single-threaded spinning. For example, if you are using an AsyncSpinner, callbacks are called from multiple threads and additional care needs to be taken.


Originally posted by prex on ROS Answers with karma: 151 on 2020-12-08

Post score: 2

$\endgroup$

1 Answer 1

0
$\begingroup$

Rosanswers logo

After spending quite some time looking for information about ROS callback queues, multi-threaded spinners and timers, I think I can answer my questions. Seems like the ROS documentation could be improved on this side.

A very useful article about multi-threaded spinners and multiple callback queues can be found here: https://levelup.gitconnected.com/ros-spinning-threading-queuing-aac9c0a793f

As known, ROS does some internal threading. It runs each subscriber in a receiver thread and each timer in a timer thread. This allows to receive data independent of the spinner. However, these threads do not process the callback, they just collect it. For the subscriber, it adds an element to the subscriber queue and for the timer, it adds the timer callback to the callback queue. Timer callbacks end up in the same callback queue with subscriber callbacks.

Now, on each spin the subscriber queue is added to the callback queue (number of elements to keep in describer queue can be set. If spinner has much lower frequency than incoming subscriber callbacks, they might be dropped). The complete callback queue is processed in first-in-first-out order. This means in each spin, every element in the callback queue gets processed.

Callbacks should always be fast, because multiple of the same callback can end up in the queue if the subscriber queue is not limited to 1. For the timer to work properly, it is required that the spinner runs regularly at a fast frequency, because the ROS timer just adds elements to the callback queue when spin is called and might only execute it after other callbacks are processed.

Also, for this reason, it makes sense to use timers instead of limiting the frequency of the spinner. Limiting the frequency means that callbacks can not be processed anymore just because of maybe one function which should be executed at a certain frequency. But the spinner does much more and we should not limit it.

Since the spinner is a bottleneck if processing the callback queue takes too much time, it makes sense to use a multi-threaded spinner. If you use a multi-threaded spinner/ async spinner, keep in mind that a lock is applied for a specific callback (no concurrency per default) and multiple callbacks of the same type are not processed in parallel. However, if enough threads are available, the next unlocked callback of the queue will be called (https://stackoverflow.com/a/48544551/8623933) (https://roboticsbackend.com/ros-asyncspinner-example/ ). This also means that if a timer callback takes longer than the timer duration, it will start with a delay. No new timer callback will be added to the callback queue during this time (https://answers.ros.org/question/248656/does-callbacks-get-drop-from-queue-when-it-is-exceeded-it-expected-execution-time-by-too-much/).

For the case that there are processes which are more time critical and can not wait until the elements in front of the callback queue are processed, we can use multiple callback queues and assign subscribers and timers (http://docs.ros.org/en/diamondback/api/roscpp/html/structros_1_1TimerOptions.html) to them. There are different options of doing this. The gitconnected article linked above describes how to create another spinner in a new thread. Another option is to add an AsyncSpinner with its own callback queue in addition to a global callback queue as shown here: https://gist.github.com/bgromov/e6f5eb142346b3c88e9f96bce17eee92

So as a summary, the solution to my problem is to create an AsyncSpinner, which can process the timer callback in parallel so that it does not block the faster callbacks. If this is not enough, I would start using multiple callback queues for critical processes.

Two more comments about spinners and how they process the callback queue:

  • There are 2 steps for a spinner to execute a callback from the callback queue. First, it needs to get the callback from the queue, and second, it executes the callback. During the first step, there's a lock on the callback queue so that no other threads can access it at the same time. But once the callback is loaded, another callback can be loaded into another thread. Step 2 works in parallel. (Explained by the author of the gitconnect article)

  • The AsyncSpinner should be able to continue executing a new spin (begin with a new callback queue) in parallel if a thread is available, even if one callback takes much more time. (See comment here: https://stackoverflow.com/a/48544551/8623933)


Originally posted by prex with karma: 151 on 2020-12-10

This answer was ACCEPTED on the original site

Post score: 7


Original comments

Comment by jarvisschultz on 2020-12-10:
Very helpful write-up! Thanks for doing some research. Couple of quick notes:

  1. The threading model is quite different in rospy https://answers.ros.org/question/9543/rospy-threading-model/
  2. Agree that the documentation on these things is not great. Reminder, this documentation is a wiki. Feel free to add your own contributions and edits.

Comment by prex on 2020-12-10:
Thanks @jarvisschultz. I added the hint that I'm talking about roscpp.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.