I am running a demo ROS2 talker on a FreeBSD-based OS on QEMU. It is called minimal_publisher and is publishing to chatter. The talker is from demo_nodes_cpp provided by the vanilla install of ROS2 dashing.
I then run the corresponding listener on the host machine, subscribing to chatter.
The talker on QEMU looks like its publishing just fine (printing a string of Publishing: 'Hello, world! [count]'; however, the listener doesn't pick anything up.
ros2 node list shows both nodes, but ros2 topic list does not show the chatter topic. It only shows parameter_events and rosout.
This setup worked for me with ROS[1], running roscore on the host. In that case, though, I was able to explicitly point the talker running on QEMU to the master via ROS_HOSTNAME and ROS_MASTER_UID, I think.
I would appreciate some help in getting my ROS2 publishers and subscribers similarly working across the QEMU boundary.
Originally posted by broomstick on ROS Answers with karma: 111 on 2020-04-11
Post score: 0
Original comments
Comment by gvdhoorn on 2020-04-11:
First thing to check: make sure UDP multicast is working between all involved hosts/emus.
See the first part of ROS 2 Overview > Troubleshooting to check. Also: Getting started with ros2doctor.
Comment by broomstick on 2020-04-11:
The troubleshooting recommends running ros2 multicast and, if that fails, then modifying firewall rules and checking for MULTICAST tags via ifconfig.
My target environment doesn't support python (yet), so I'm not able to build and use ros2cli (unless I'm mistaken, which would make me happy...). I can send and receive multicast packets on the host (via ros2 multicast), but I can't run ros2 multicast send in the QEMU environment and ros2 multicast receive on the host...
ifconfig does show MULTICAST in the flags section of both interfaces displayed by ifconfig on QEMU and I've confirmed that the firewall is not enabled.
To get this demo going, is there a way to simplify this process by reverting to something closer to the ROS1 discovery/communication paradigm? Eventually, I want to be able to use the fully featured, distributed discovery provided by ROS2, but I need to get some demos up and running to justify the time!
Comment by gvdhoorn on 2020-04-11:
You could see whether configuring things for unicast discovery works. There are a couple of Q&As on this site about that. It comes down to editing the Fast RTPS configuration to disable multicast.
There's no guarantee of course that this is your problem. It was just the first thing I would check.
Also: you don't absolutely need to use the ros2 multicast verb. Any tool which can check multicast capabilities could be used. Software based routers/interfaces sometimes don't implement the necessary routing (IGMP snooping, etc). It could be that's what happening here, but it could be something else completely as well.
Comment by broomstick on 2020-04-11:
I ran a fairly quick test based on this code, which (I think) demos multicast capability. It worked just fine back and forth between the host and QEMU.
I also tried to establish unicast discovery based on this ROS answer and eProsima documentation (see section on Initial Peers and subsequent tip on disabling multicast). I've posted my DEFAULT_FASTRTPS_PROFILES.xml file.
Still, when I run the talker from within QEMU, it shows up on the host in response to ros2 node list, but the topic does not appear when I run ros2 topic list on the host.
Comment by broomstick on 2020-04-11:
Some other observations:
ros2 node info /minimal_publisher produces the following error:
ros2_dashing $ ros2 node info /minimal_publisher
/minimal_publisher
[ERROR] [rmw_fastrtps_shared_cpp]: Unable to find GUID for node: minimal_publisher
Failed to get_subscriber_names_and_types: Unable to find GUID for node , at /home/broomstick/ros2_dashing/src/ros2/rmw_fastrtps/rmw_fastrtps_shared_cpp/src/rmw_node_info_and_types.cpp:87
Comment by broomstick on 2020-04-11:
If I run the talker and then the listener on QEMU (or visa versa), the second one fails with the following error:
1586640423.685765 [42] 100051: UDP make_socket failed for multicast port 17900
[ERROR] [1586640423.781729296] [rmw_cyclonedds_cpp]: rmw_create_node: failed to create domain, error Error
>>> [rcutils|error_handling.c:108] rcutils_set_error_state()
This error state is being overwritten:
'error not set, at /home/broomstick/cheri/ros2_min/src/ros2/rcl/rcl/src/rcl/node.c:336'
with this new error message:
'rcl node's rmw handle is invalid, at /home/broomstick/cheri/ros2_min/src/ros2/rcl/rcl/src/rcl/node.c:485'
rcutils_reset_error() should be called after error handling to avoid this.
<<<
[ERROR] [1586640423.895518056] [rcl]: Failed to fini publisher for node: 1
Terminating due to uncaught exception 0x48960300 of type rclcpp::exceptions::RCLError
Abort
Comment by gvdhoorn on 2020-04-12:
It does look like something isn't behaving as it should. Particularly in the network stack/interfacing code.
Seeing your other questions: could this be caused by your compiler infrastructure? Have you tried running any of the examples/tests that come with the DDS implementation? That would be one/a few layers of abstraction less, while still being more complex than a bare multicast test.
There is always the chance it's actually a problem in the ROS 2 code, but that's not entirely apparent right now.
Comment by broomstick on 2020-04-13:
As I'm currently building with -DBUILD_TESTING=NO, the tests in rmw don't get built. I've tried to find an easy workaround for this, but it'll take some time, as I also need to build the google test suite.
To avoid communicating host-> guest, I tried running one of the composed publisher/subscribers. It seems to do better, but I get the following when I initiate. As you can see, the subscriber prints an empty message:
Comment by broomstick on 2020-04-13:
Error message:
[INFO] [1586660033.578303216] [publisher_node]: Publisher: 'Hello, world! 0'
>>> [rcutils|error_handling.c:108] rcutils_set_error_state()
This error state is being overwritten:
'Security was requested but the Cyclone DDS being used does not have security support enabled. Recompile Cyclone DDS with the '-DENABLE_SECURITY=ON' CMake option, at /home/broomstick/cheri/ros2_min/src/ros2/rmw_cyclonedds/rmw_cyclonedds_cpp/src/rmw_node.cpp:834'
with this new error message:
'invalid data size, at /home/broomstick/cheri/ros2_min_no_cheri/src/ros2/rmw_cyclonedds/rmw_cyclonedds_cpp/src/serdata.cpp:308'
rcutils_reset_error() should be called after error handling to avoid this.
<<<
[INFO] [1586660033.973517976] [subscriber_node]: Subscriber: ''
Comment by gvdhoorn on 2020-04-13:\
As I'm currently building with
-DBUILD_TESTING=NO, the tests inrmwdon't get built. I've tried to find an easy workaround for this, but it'll take some time, as I also need to build the google test suite.
RMW != DDS implementation.
I'm referring to the plain distribution provide by the DDS vendor. Those also come with test / example binaries.
Comment by broomstick on 2020-04-14:
Running the cyclonedds examples seems to require Apache Maven (to make use of the raw idl files, I think?), which I can't build on my target system at the moment.
While eProssima's Fast-RTPS also has idl files, it doesn't appear to have the same Maven dependency.
I'm having trouble getting Fast-RTPS installed, however, as you can see from a separate question I just posted. I also switched to Eloquent to try to make use of ros2 doctor, as well as some other changes.
I'll come back to this question once I've got some examples run.
Comment by gvdhoorn on 2020-04-14:
Seeing as Cyclone is OSS, you may want to reach out to them on their issue tracker. They've been very accommodating in the past.
At least with Fast RTPS the Java dependency is fully optional. I would expect it to be the same with Cyclone, but that would be something to ask them.
Comment by broomstick on 2020-04-14:
Posted here.