0
$\begingroup$

I am a ros2 (humble) based robotics project on the Raspberry pi 4b. Due to personal constraints, I carried out a headless installation of the pi. To use ROS2 efficiently, I created a virtual machine (using vmare pro) that runs ubuntu 22.04LTS, while I did the SSH connection on the host machine (windows 11). I have set up all my stuff - the workspace, th e packages, etc. The problem now is that say I start up my camera and the object detection launch files, I can see it work but the moment I go to my vm to use the visual features to see the images sent over the topic, my ssh connction disconnects. I believe I should point out that I had a very hard time setting up a bridge network on the vm (that allows me to see the ros2 topics on the local nework), so I found a youtube video that taught me to hardcode the ip address, submask, etc for it to work. I have confirmed that it is noit the ip addresses clashing, everything runs fine until I open up say, rqt_image_view and then things start misbehaving. It has worked a few times, and in those times, it was just a single launch file running at a time. Please help. Thank you.

On Vscode (ssh remote): I run the camera launch file:

def generate_launch_description():
    return LaunchDescription([
        # Static transform from base_footprint to base_link
        Node(
            package='v4l2_camera',
            executable='v4l2_camera_node',
            output='screen',
            parameters=[{'image_size': [640, 480]},
            {'camera_frame_id': 'camera_link_optical'}
            ] 
        ),
    ])

The object_detection node:

class YOLOv8ObjectDetector(Node):
    def __init__(self):
        super().__init__('yolov8_object_detector')
        self.bridge = CvBridge()
        package_dir = get_package_share_directory('yolo')  # Replace with your package name
        weights_path = os.path.join(package_dir, 'weights', 'best.pt')

        self.model = YOLO(weights_path)  # Path to your trained YOLOv8 weights
        self.subscription = self.create_subscription(
            Image,
            '/image_raw',
            self.listener_callback,
            10)
        self.publisher_ = self.create_publisher(Image, '/yolov8_detected_image', 10)

    def listener_callback(self, msg):
            self.get_logger().info('Receiving video frame')
            current_frame = self.bridge.imgmsg_to_cv2(msg, "bgr8")

            results = self.model(current_frame)
            boxes = results[0].boxes

            for box in boxes:
                # Get the coordinates
                xyxy = box.xyxy[0].tolist()
                x1, y1, x2, y2 = int(xyxy[0]), int(xyxy[1]), int(xyxy[2]), int(xyxy[3])

                # Get the class ID and confidence
                cls = int(box.cls[0])
                conf = float(box.conf[0])  # Convert tensor to float

                # Get the label
                label = f'{self.model.names[cls]} {conf:.2f}'

                # Draw the bounding box and label on the image
                cv2.rectangle(current_frame, (x1, y1), (x2, y2), (255, 0, 0), 2)
                cv2.putText(current_frame, label, (x1, y1 - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 0, 0), 2)

            # Convert the annotated image back to ROS Image message
            annotated_msg = self.bridge.cv2_to_imgmsg(current_frame, "bgr8")

            # Publish the annotated image
            self.publisher_.publish(annotated_msg)
def main(args=None):
    rclpy.init(args=args)
    yolov8_object_detector = YOLOv8ObjectDetector()
    rclpy.spin(yolov8_object_detector)
    yolov8_object_detector.destroy_node()
    rclpy.shutdown()

if __name__ == '__main__':
    main()

I run the two codes above, and they work fine. I start getting object detection readings. I then head over to my VM and on a terminal I run:

''' ros2 run rqt_image_view rqt_image_view '''

The moment it opens, the ssh connection on visual studio (on host machine) starts reconnecting, which spoils everything.

$\endgroup$
5
  • $\begingroup$ " fine until I open up say, rqt_image_view and then things start misbehaving" Please edit your question to include examples of exactly what you're running and exactly the errors that you're encountering. Without those, it could be almost anything. An MRE would be best. $\endgroup$
    – Tully
    Commented Jul 2 at 15:26
  • $\begingroup$ My bad. Thank you. Edited. $\endgroup$ Commented Jul 3 at 2:07
  • $\begingroup$ How much network bandwidth do you have between the devices? If the ssh connection is dropping that is a networking connection issue. And X forwarding of a gui takes up a lot of bandwidth, potentially overwhelming network buffers if there's not enough bandwidth. $\endgroup$
    – Tully
    Commented Jul 3 at 16:10
  • $\begingroup$ I see... I currently do not know how to measure the network bandwidth I have between the devices. I use a simple 4G LTE router that the devices are connected to. $\endgroup$ Commented Jul 3 at 22:21
  • $\begingroup$ You're going to want to look closer at that and your network topology to find what are your limitations there as well as on the raspberry pi networking stack and make sure not to overwhelm it with the data you're trying to stream, especially if you're on wifi which is unclear from the description. $\endgroup$
    – Tully
    Commented Jul 4 at 17:07

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.