1
$\begingroup$

Below is an example of how to use the ROSBag API to extract image data from a ROSBag file, and then visualize that data using OpenCV.

#include <ros/ros.h>  
#include <rosbag/bag.h>  
#include <rosbag/view.h>  
#include <sensor_msgs/Image.h>  
#include <opencv2/opencv.hpp>  
#include <cv_bridge/cv_bridge.h>  
  
int main(int argc, char** argv){  
    ros::init(argc, argv, "image_extractor");  
    rosbag::Bag bag;  
    try{  
        bag.open("test.bag", rosbag::bagmode::Read);  
    }  
    catch(rosbag::BagException& e){  
        ROS_ERROR("Could not open file: %s", e.what());  
        return -1;  
    }  
  
    std::vector<std::string> topics;  
    topics.push_back(std::string("camera/image_raw"));  
  
    rosbag::View view(bag, rosbag::TopicQuery(topics));  
  
    cv::namedWindow("view", cv::WINDOW_NORMAL);  
    cv::startWindowThread();  
  
    BOOST_FOREACH(rosbag::MessageInstance const m, view){  
        sensor_msgs::Image::ConstPtr img_msg = m.instantiate<sensor_msgs::Image>();  
        if(img_msg != NULL){  
            cv_bridge::CvImageConstPtr cv_ptr;  
            try{  
                cv_ptr = cv_bridge::toCvShare(img_msg, "bgr8");  
            }  
            catch (cv_bridge::Exception& e){  
                ROS_ERROR("cv_bridge exception: %s", e.what());  
                continue;  
            }  
            cv::imshow("view", cv_ptr->image);  
            cv::waitKey(30);  
        }  
    }  
  
    cv::destroyWindow("view");  
    bag.close();  
  
    return 0;  
}  

Now my question is, how do I do that if the images are compressed in h264 format?

$\endgroup$

1 Answer 1

0
$\begingroup$

There needs to be an h264 decoder that is instantiated at the beginning then each h264 compressed image message payload of bytes needs to be inserted into the decoder, and after a key frame is found it should be able to output individual images (but figure on the first n frames not being usable).

ffmpeg is the default choice for the decoder, and take a look at https://github.com/daniilidis-group/ffmpeg_image_transport_tools, https://github.com/daniilidis-group/ffmpeg_image_transport_tools/blob/master/src/decode_bag.cpp in particular may be nearly what you want.

I haven't tried that exactly but I have used https://github.com/daniilidis-group/ffmpeg_image_transport (which has been converted to an archive and no longer maintained)- it worked sometimes but I think was very sensitive to encoding parameters that weren't as expected or dropped or out-of-order frames- it crashed occasionally unfortunately, maybe could be forked and fixed up with some effort and knowledge of ffmpeg.

I had some luck a few months back doing something similar in python using the pytorch-audio wrapper around ffmpeg, it seemed stable but was still very inconvenient for quickly reviewing bags so I gave up on h264/5 as a ros bag format for now.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.