1
$\begingroup$

This is using ros2 Humble in Ubuntu 22.04 and native Python 3.10

Eventual goal: capturing depth aligned RGB and Depth streams from Intel RealSense D435 camera and converting them to RGB and Depth videos. Example script for ros (v1) is provided below but need a tool/script for ros2. In order to debug this problem, I did try to reduce it to only dealing with color raw topic.

I have created a bag file for this topic /camera/camera/color/image_raw and I want to convert the capture that I did with Intel RealSense D435 camera to a series of consecutive photos or preferably a video without the loss to the quality of captured data.

I have inspected my bag file and there is no problem in it. I also have used the ros2 echo command and I am able to see there is content in it.

Initially, I used the ros2bag2video.py script from this git repo but it gets stuck after running the command while echoing the bag doesn't get stuck. I also came across solution for ros1 for doing this but since I am new to ros2 I don't know how to do it for this version.

(base) mona@ada:~/rosbag2video$ /usr/bin/python3.10 ros2bag2video.py -v -t /camera/camera/color/image_raw ~/ros2bagmona/
Using  25  FPS
FPS (int) =  25
Rate (float) =  1.0
Topic (str) =  /camera/camera/color/image_raw
Output File (str) =  output.mp4
Verbose (bool) =  True
Reading info from bag file: /home/mona/ros2bagmona/
ROS Message name =  sensor_msgs/msg/Image
Image count =  198
msgtype =  <class 'sensor_msgs.msg._image.Image'>
AJB: subscribing to msg:  <class 'sensor_msgs.msg._image.Image'> on topic:  /camera/camera/color/image_raw
Starting ROS bag playback...
[INFO] [1699041025.599860882] [rosbag2_storage]: Opened database '/home/mona/ros2bagmona/ros2bagmona_0.db3' for READ_ONLY.
[INFO] [1699041025.599920787] [rosbag2_player]: Set rate to 1
[INFO] [1699041025.602356503] [rosbag2_player]: Adding keyboard callbacks.
[INFO] [1699041025.602398298] [rosbag2_player]: Press SPACE for Pause/Resume
[INFO] [1699041025.602405008] [rosbag2_player]: Press CURSOR_RIGHT for Play Next Message
[INFO] [1699041025.602410516] [rosbag2_player]: Press CURSOR_UP for Increase Rate 10%
[INFO] [1699041025.602414936] [rosbag2_player]: Press CURSOR_DOWN for Decrease Rate 10%
[INFO] [1699041025.602892229] [rosbag2_storage]: Opened database '/home/mona/ros2bagmona/ros2bagmona_0.db3' for READ_ONLY.

^^^ gets stuck executing the command after READ_ONLY message.

This is despite being able to play the bag with no problem:

(base) mona@ada:~$  ros2 bag play ros2bagmona/
[INFO] [1699041088.759139678] [rosbag2_storage]: Opened database 'ros2bagmona/ros2bagmona_0.db3' for READ_ONLY.
[INFO] [1699041088.759203803] [rosbag2_player]: Set rate to 1
[INFO] [1699041088.762416056] [rosbag2_player]: Adding keyboard callbacks.
[INFO] [1699041088.762461205] [rosbag2_player]: Press SPACE for Pause/Resume
[INFO] [1699041088.762468691] [rosbag2_player]: Press CURSOR_RIGHT for Play Next Message
[INFO] [1699041088.762474209] [rosbag2_player]: Press CURSOR_UP for Increase Rate 10%
[INFO] [1699041088.762478606] [rosbag2_player]: Press CURSOR_DOWN for Decrease Rate 10%
[INFO] [1699041088.762974737] [rosbag2_storage]: Opened database 'ros2bagmona/ros2bagmona_0.db3' for READ_ONLY.
(base) mona@ada:~$

So, either can you show a solution for how to correctly use ros2bag2video.py for converting a bag file of raw color to rgb video or a series of consecutive photos using any other method that would be compatible with ros2 humble bags?

Also, I am not sure if there are certain version of opencv-python that are compatible with cv-bridge package, but here's the package versions I have:

(base) mona@ada:~$ /usr/bin/pip show cv-bridge
Name: cv-bridge
Version: 3.2.1
Summary: UNKNOWN
Home-page: UNKNOWN
Author: 
Author-email: 
License: UNKNOWN
Location: /opt/ros/humble/local/lib/python3.10/dist-packages
Requires: 
Required-by:

and

(base) mona@ada:~$ /usr/bin/pip show opencv-python
Name: opencv-python
Version: 4.5.5.62
Summary: Wrapper package for OpenCV python bindings.
Home-page: https://github.com/skvark/opencv-python
Author: 
Author-email: 
License: MIT
Location: /home/mona/.local/lib/python3.10/site-packages
Requires: numpy, numpy, numpy, numpy
Required-by: 


(base) mona@ada:~$ ros2 wtf
/opt/ros/humble/lib/python3.10/site-packages/ros2doctor/api/package.py: 112: UserWarning: joy has been updated to a new version. local: 3.1.0 < latest: 3.3.0
/opt/ros/humble/lib/python3.10/site-packages/ros2doctor/api/package.py: 112: UserWarning: sdl2_vendor has been updated to a new version. local: 3.1.0 < latest: 3.3.0

All 5 checks passed

Also, I have seen this script but not sure what's the equivalent for ros2. Is there a tool for converting code written for ros to ros2?

import os
import sys
import yaml
from rosbag.bag import Bag
import cv2

import roslib;   #roslib.load_manifest(PKG)
import rosbag
import rospy
import cv2
import numpy as np
import argparse
import os

from sensor_msgs.msg import Image
from cv_bridge import CvBridge
from cv_bridge import CvBridgeError

class ImageCreator():
    def __init__(self, bagfile, rgbpath, depthpath, rgbstamp, depthstamp):
        self.bridge = CvBridge()
        with rosbag.Bag(bagfile, 'r') as bag:
            for topic,msg,t in bag.read_messages():
                print(topic)
                if topic == "/camera/color/image_raw": #图像的topic；
                    # try:
                    cv_image = self.bridge.imgmsg_to_cv2(msg,"bgr8")
                    # except CvBridgeError as e:
                    #     print(e)
                    timestr = "%.6f" %  msg.header.stamp.to_sec()
                    #%.6f表示小数点后带有6位，可根据精确度需要修改；
                    image_name = timestr+ ".png" #图像命名：时间戳.png
                    # cv2.imshow("color", cv_image)
                    cv2.waitKey(1);
                    cv2.imwrite(rgbpath + image_name, cv_image)  #保存；

                    # #写入时间戳
                    # with open(rgbstamp, 'a') as rgb_time_file:
                    #     rgb_time_file.write(timestr+" rgb/"+image_name+"\n")
                elif topic == "/camera/aligned_depth_to_color/image_raw": #图像的topic；
                    # try:
                    cv_image = self.bridge.imgmsg_to_cv2(msg,"16UC1")
                    # except CvBridgeError as e:
                    #     print(e)
                    timestr = "%.6f" %  msg.header.stamp.to_sec()
                    #%.6f表示小数点后带有6位，可根据精确度需要修改；
                    image_name = timestr+ ".png" #图像命名：时间戳.png
                    
                    # cv2.imwrite(depthpath + image_name, cv_image)  #保存；
                    cv2.imwrite(depthpath + image_name, (cv_image).astype(np.uint16))

                    # #写入时间戳
                    # with open(depthstamp, 'a') as depth_time_file:
                    #     depth_time_file.write(timestr+" depth/"+image_name+"\n")

if __name__ == '__main__':
    ImageCreator('cheezit_new_2.bag', "/home/sc19zx/rgb/", "/home/sc19zx/depth/", 1, 1)

^^^ From https://github.com/NVlabs/BundleSDF/issues/82#issuecomment-1792602336

(base) mona@ada:~$ /usr/bin/python3.10
Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] on linux


(base) mona@ada:~$ /usr/bin/pip --version
pip 22.0.2 from /usr/lib/python3/dist-packages/pip (python 3.10)


(base) mona@ada:~$ uname -a
Linux ada 6.2.0-36-generic #37~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Oct  9 15:34:04 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
(base) mona@ada:~$ lsb_release -a
LSB Version:    core-11.1.0ubuntu4-noarch:security-11.1.0ubuntu4-noarch
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.3 LTS
Release:    22.04
Codename:   jammy

I have checked both depth and color in rviz2 and am able to see them. Further, I have done depth aligned capture.

(base) mona@ada:~$ ros2 bag info ros2bagmona/

Files:             ros2bagmona_0.db3
Bag size:          174.2 MiB
Storage id:        sqlite3
Duration:          13.142s
Start:             Nov  2 2023 11:52:10.972 (1698940330.972)
End:               Nov  2 2023 11:52:24.115 (1698940344.115)
Messages:          198
Topic information: Topic: /camera/camera/color/image_raw | Type: sensor_msgs/msg/Image | Count: 198 | Serialization Format: cdr

(base) mona@ada:~$ ls ros2bagmona/
total 175M
-rw-r--r--  1 mona mona 175M Nov  2 11:52 ros2bagmona_0.db3
drwxrwxr-x  2 mona mona 4.0K Nov  2 11:52 .
-rw-rw-r--  1 mona mona 1005 Nov  2 11:52 metadata.yaml
drwxr-x--- 61 mona mona 4.0K Nov  3 15:44 ..

In case it may be important, I am using this version of package:

(base) mona@ada:~/rosbag2video$ git log
commit 13b4e064eb0ad748f3d4e20eb88353bdf76acd21 (HEAD -> master, origin/master, origin/HEAD)
Merge: 44d1637 14a7d26
Author: mlaiacker <[email protected]>
Date:   Mon Sep 18 20:11:28 2023 +0200

    Merge branch 'pipebots-humble-dev-joey'

and I have made a git issue here.

Improve this question
$\endgroup$
7
  • $\begingroup$ Does it work on this small ros2 bag github.com/lucasw/ros_one2z/files/13256937/…? (It's in mcap format not db3 though) I had to change img = self.bridge.compressed_imgmsg_to_cv2(msg, self.msg_fmt) to img = self.bridge.imgmsg_to_cv2(msg, self.msg_fmt) to make it work with uncompressed raw images $\endgroup$
    – Lucas Walter
    Commented Nov 4, 2023 at 14:52
  • 1
    $\begingroup$ 'it' -> ros2bag2video $\endgroup$
    – Lucas Walter
    Commented Nov 7, 2023 at 14:40
  • 1
    $\begingroup$ Are you able to share the db3 ros2 bag? $\endgroup$
    – Lucas Walter
    Commented Nov 7, 2023 at 14:42
  • 1
    $\begingroup$ Ok if you have something that works for non depth images go ahead and make an answer here, put in the details $\endgroup$
    – Lucas Walter
    Commented Nov 8, 2023 at 17:16
  • 1
    $\begingroup$ I added the answer. Thank you $\endgroup$
    – Mona Jalal
    Commented Nov 8, 2023 at 18:10

2 Answers 2

Reset to default
0
$\begingroup$

using https://gitlab.com/ternaris/rosbags and only for color raw topic

from pathlib import Path

from rosbags.highlevel import AnyReader
import numpy as np
from PIL import Image
from datetime import datetime
from matplotlib import image
import cv2
import os
import shutil

import matplotlib.pyplot as plt



if os.path.exists(os.path.join(os.getcwd(), 'rgb')):
    shutil.rmtree(os.path.join(os.getcwd(), 'rgb'))
    os.mkdir(os.path.join(os.getcwd(), 'rgb'))
             

# create reader instance and open for reading
with AnyReader([Path('/home/mona/rosbag2_2023_11_06-15_44_24')]) as reader:

    connections = [x for x in reader.connections if x.topic == '/camera/camera/color/image_raw']
    for connection, timestamp, rawdata in reader.messages(connections=connections):
        msg = reader.deserialize(rawdata, connection.msgtype)
        timestamp_dt = datetime.fromtimestamp(msg.header.stamp.sec + msg.header.stamp.nanosec * 1e-9)
        timestamp_str = timestamp_dt.strftime("%Y-%m-%d %H:%M:%S.%f")
        timestamp_ns = msg.header.stamp.sec * 1e9 + msg.header.stamp.nanosec
        numeric_timestamp = int(timestamp_ns / 1e-9)
        
        w, h = msg.width, msg.height
    
        color_image_data = np.reshape(msg.data, (h, w, 3)) # for rgb image
        rgb_image_name = 'rgb/' + str(numeric_timestamp)[:20] + '.png'
        image = Image.fromarray(color_image_data)
        image.save(rgb_image_name)
Improve this answer
$\endgroup$
0
$\begingroup$

I briefly googled about rosbag2video and found this fork https://github.com/mlaiacker/rosbag2video which seems support Humble. Give it a try.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.