Rosanswers logo

I'm attempting to record short (~10s) video streams from a kinect. Ideally I'd like to output two video files after recording is finished: one file for RGB and the other file for depth (I can save a series of images rather than video file if that's not possible, but that tends to be space intensive, so I'd prefer not to do that). This is so I can process the data later. I have been searching for the past few days around the internet, and while I have found many resources and posts that are close to what I want to do, none of them are quite exactly what I'm looking for. I'm writing in Python, which is relevant since a lot of the posts I've found refer to C++ code, but I can never seem to find the equivalent function calls for Python.

Here's what I've got at the moment, but it is very hacky and doesn't really work. For recording the RGB data, I use image_view extract_images. I make this function call:

subprocess.Popen(["rosrun","image_view","extract_images","_sec_per_frame:=0.01","image:=camera/rgb/image_color"], cwd="temp/rgb/", stdout=DEVNULL)

To record the depth data, I wrote a class that subscribes to the "/camera/depth/image_rect" topic and then writes out the images to files:

class ImageConverter:


def __init__(self):
    self.count = 0
    self.todo = Queue.Queue()
    self.stop = False
    self.sub = None
    self.threads = None
    self.countLock = threading.Lock()
def __getCounter(self):
    c = self.count
    self.count = self.count + 1
    ret = str(c)
    while len(ret) < 4:
        ret = "0" + ret
    return ret

def __callback(self, data):
def __processImage(self,data):
    bridge = CvBridge()
        source = bridge.imgmsg_to_cv(data, "32FC1")
    except CvBridgeError, e:
        print e
    ret = cv.CreateImage((data.width, data.height), 8, 3)
    mx = 0.0
    for i in range(data.width):
        for j in range(data.height):
            d = source[j,i]
            if d > mx:
                mx = d
    for i in range(source.width):
        for j in range(source.height):
            v = source[j,i]/mx*255
            ret[j,i] = (v,v,v)
    return ret
def startRecording(self,topic):
    self.stop = False
    def processor():
        rate = rospy.Rate(100.0)
        while (not self.stop) or (not self.todo.empty()):
            if not self.todo.empty():
                data = self.todo.get()
                img = self.__processImage(data)
                c = self.__getCounter()
                cv.SaveImage("depth/frame" + c + ".jpg", img)
                print("[ImageConverter]: image saved " + c + "/" + str(self.todo.qsize()))
    self.threads = []
    for i in range(ImageConverter.NUM_THREADS):
        t = threading.Thread(target=processor, args = ())
        t.daemon = True
    self.sub = rospy.Subscriber(topic, Image, self.__callback)
def stopRecording(self):
    self.stop = True
    for t in self.threads:

As you can see in the __processImage method, I had to manually convert the image from 32FC1 to bgr8, since everytime I try to call the cv_bridge method imgmsg_to_cv with 'bgr8' (or 'mono8' for that matter) it throws an error saying it can't convert and saving directly as 32FC1 yields images that are thresholded to either max brightness or black. This is of course very slow, which is why I spin the processing off into a separate thread from the callback so I don't drop images.

After saving the series of images, I call mencoder to compile them into a video file, which usually reduces the total size down to about a fifth of what all the individual images take up:

subprocess.Popen("mencoder \"mf://*.jpg\" -mf type=jpg:fps=24 -o " + filename + " -speed 1 -ofps 24 -ovc lavc -lavcopts vcodec=mpeg2video:vbitrate=2500 -oac copy -of mpeg", cwd="temp/rgb/", shell=True, close_fds=True, stdout=DEVNULL, stderr=DEVNULL)

This seems like a pretty big hack. It seems like there should be a better way to do this, but after a few days of searching through forums and other ROS resources, I can't seem to find anything that can help. Additionally, the video files I get out of this are usually jumpy and sometimes frames are entirely missing. Is there any good way to record both RGB and Depth data as video files or a series of images using Python in ROS?

Originally posted by rse101 on ROS Answers with karma: 36 on 2014-02-22

Post score: 1


2 Answers 2


Rosanswers logo

maybe rosbag

Originally posted by Hamid Didari with karma: 1769 on 2014-02-22

This answer was NOT ACCEPTED on the original site

Post score: 3

Original comments

Comment by Dan Lazewatsky on 2014-02-23:
What I've done in the past is record with rosbag, and I wrote a utility for generating video files from bags: https://github.com/OSUrobotics/bag2video

Comment by rse101 on 2014-02-23:
I've looked at using rosbag, but it seems like it only pushes the problem down the line. For example, this (http://wiki.ros.org/rosbag/Tutorials/Exporting%20image%20and%20video%20data) tutorial explains how to extract video from a rosbag, but why can't I just record straight from the topic rather than storing in a rosbag first? Actually, that tutorial is the reason I made the calls to 'extract_images' and 'mencoder' in the first place, but I didn't see any reason to store in a rosbag and then play back. Dan, thanks for the utility. When I get back into the lab I will see if I can get that to work. If I can, I think I might go with this route.

Comment by Dan Lazewatsky on 2014-02-23:
Yeah, I think the real answer is that what you're trying to do is pretty common, but there's no standard solution. It used to be possible with image_saver + mencoder/ffmpeg, but image_saver disappeared.

Comment by rse101 on 2014-02-24:
Thanks for your code. I ran it and it works great for the RGB data, but I get the same problem with depth data as in my original post. I get an error thrown at line 40 of bag2video.py: img = np.asarray(bridge.imgmsg_to_cv(msg, 'bgr8')) that OpenCV can't convert from 32FC1 to bgr8. I found another solution which I'll write up, but thanks anyways.

Comment by Dan Lazewatsky on 2014-02-25:
Hmm, I had never tried with depth. Even though you found another solution, next time I have a Kinect handy, I'm going to poke around and figure out what the correct conversion is.

Comment by aguadopd on 2014-03-01:
@rse101, how did you manage to save depth images? I cant see them with image_view (however they are working in RVIZ)

Comment by rse101 on 2014-03-02:
Sorry about that. I originally posted my answer a few days ago, but ROS Answers wouldn't post it and it wasn't until today that I figured out that it was because I have links in the answer. I managed to get it posted now, so hopefully that answers your question.

Comment by aguadopd on 2014-03-04:
@rse101, thanks for your answer. I've just found the video_record tool here: http://wiki.ros.org/image_view . It may work if you alrady can see that topics with image_view.


Rosanswers logo

Ok, so the solution I found is a little specific to my project. I determined that OpenNI .oni files will work as video file output for my purposes. If anyone is reading this and wants to go all the way to .avi files, if you try my solution and then add this:


(replace '[dot]' with '.') solution on top of that, you should be able to get to .avi files.

To get the .oni files from the RGB and depth data, I first downloaded OpenNI (okay, so I already had OpenNI from before, but the ROS version isn't the same as the release version, so I downloaded that and put it in it's own folder) from here:


(replace '[dot]' with '.') (Note, I'm running Linux, so this may work slightly differently for you if you are running something else). I extracted the folder and just placed it in my home directory (so it didn't interfere with ROS's OpenNI). Next I downloaded Primesense:


(replace '[dot]' with '.') which adds python bindings for OpenNI. I ran

setup.py build


setup.py install

from the Primesense folder to install the necessary libraries. I then ran the following python code to record the .oni files:

import roslib
import rospy
from primesense import openni2


dev = openni2.Device.open_any()

depth_stream = dev.create_depth_stream()
color_stream = dev.create_color_stream()
rec = openni2.Recorder("test.oni")
print rec.start()

#Do stuff here



where OPENNI_REDIST_DIR is the path to the redist folder in the OpenNI library. You can play these back just to double-check that they worked by running 'NiViewer' (in the Tools folder of the OpenNI library) with the path to the .oni file.

Originally posted by rse101 with karma: 36 on 2014-03-02

This answer was ACCEPTED on the original site

Post score: 0


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.