ROS1 stop sending all messages for seconds

Question

Image worth a thousand words. At arbitrary time all the messages in all topics are stop sending for ~2 sec. without apparent reason.

CPU loading at the moment was not highest - about 70% on few CPUs and 0% on others. After the gap, sending continued without delays as before. One observation: after the gap, header.seq for all messages continued normally without lose. E.g. for one message the last header.seq before gap was 1031 and the next one after gap was 1032 - nothing lost. So I don't think it's a rosbag problem. Most likely it's a ROS problem.

I understand it's barely possible to say exactly what happened. I'm rather looking for advice on how to investigate it and what direction to dig into here: what are possible reasons, what else to check?

Tech details: ROS1 Melodic, Ubuntu 18, Hardware: 8x3hz CPU + GPU + 18GB RAM

---

Originally posted by mch on ROS Answers with karma: 46 on 2019-07-16

Post score: 0

Answer 1

I have occasionally seen issues like this before. Most of the time, it happens because sensor data is coming over a network, and the network connection is interrupted for a few seconds, usually due to a loose cable.

This could also happen because something adjusted your system clock ahead in time by two seconds.

Time adjustments are logged to /var/log/syslog , and so are local network events.

I have yet to see a case where this was a "ROS problem". Topic setup is mediated by the ROS master, but topic data is carried peer-to-peer between ROS nodes, so there isn't a central point in ROS that can cause a data disruption like this.

---

Originally posted by ahendrix with karma: 47576 on 2019-07-16

This answer was ACCEPTED on the original site

Post score: 0