0
$\begingroup$

I am facing the following situation:

  1. A publisher that periodically sends a message to topic, queue size is set to 1.
  2. A subscriber (queue size 10) listening on that message, and in a callback, attempts to pass that on to a remote destination remote, through regular http request (done via python requests)
  3. remote may not always be accessible. When it is not, the requests will fail, after some delay (built-in retries/timeout/ etc.).
  4. This seems to cause the subscriber thread to error out? I then wrapped the requests in try/except blocks, which seemed to have solved the issue.
  5. However, tonight I just observed the issue again. The symptom is as follows:
  • The node running the publisher is still running, since the log message is appearing every few seconds as expected.
  • If I manually echo the topic, nothing gets printed out. So, presumably the publisher is not actually publishing the message.
  • If I try to manually publish message to topic, I do not get any response from the subscriber. Is there a way to check if the subscriber is still active? Or if the node is still running? Or if the queue is full?
  • This may be a coincident, but in the log I see 10 instances of "Error making POST request", which is my log message for when the requests to remote fails. This count of 10 matches the queue size I have set for the subscriber. However, things are wrapped in try/except blocks, so if I'm understanding correctly, the queue should still be cleared once the callback returned (even if from the except condition)?

In any case, I am scratching my head and not understanding what's going on. I can simply restart things and they will continue to work for quite some time, but as the remote comes and goes, it seems that this scenario is still going to happen unless I figure out exactly what's going on. Hence, any pointers would be much appreciated!!

$\endgroup$
6
  • $\begingroup$ Is this under ROS 1 or ROS 2? You tagged your question with both, but I think in this case it matters which version it is (I'm thinking of QoS settings). $\endgroup$ Commented Jul 7 at 15:11
  • 1
    $\begingroup$ It would also help if you showed your code. $\endgroup$ Commented Jul 7 at 15:11
  • $\begingroup$ 1. Could we see the subscriber-callback function? Try/Catch SHOULD be part of this routine, so you already did it correctly. 2. You can use the ROS2 CLI tools like ros2 node list ros2 topic list ros2 hz $topicname to check aliveness of nodes (and therefore processes). 3. You can use e.g. wiki.ubuntuusers.de/htop to check for aliveness of processes on OS-level. $\endgroup$
    – Scoeerg
    Commented Jul 8 at 13:45
  • $\begingroup$ @ChristianFritz This is ROS2 (I thought ROS was just a more general tag). I'll try to update with some minimal code from relevant parts. $\endgroup$
    – badatmath
    Commented Jul 8 at 16:08
  • $\begingroup$ @Scoeerg Thanks for the tips. I'll try to update with some minimal code from relevant parts. For now, I added timeout parameters to all the python requests I've utilized - my suspicion is that some of those may have been hanging forever and hence blocking the entire thing? I'll try things out when I encounter the issue again. $\endgroup$
    – badatmath
    Commented Jul 8 at 16:10

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.