I'm attempting to record short (~10s) video streams from a kinect. Ideally I'd like to output two video files after recording is finished: one file for RGB and the other file for depth (I can save a series of images rather than video file if that's not possible, but that tends to be space intensive, so I'd prefer not to do that). This is so I can process the data later. I have been searching for the past few days around the internet, and while I have found many resources and posts that are close to what I want to do, none of them are quite exactly what I'm looking for. I'm writing in Python, which is relevant since a lot of the posts I've found refer to C++ code, but I can never seem to find the equivalent function calls for Python.
Here's what I've got at the moment, but it is very hacky and doesn't really work. For recording the RGB data, I use image_view extract_images. I make this function call:
subprocess.Popen(["rosrun","image_view","extract_images","_sec_per_frame:=0.01","image:=camera/rgb/image_color"], cwd="temp/rgb/", stdout=DEVNULL)
To record the depth data, I wrote a class that subscribes to the "/camera/depth/image_rect" topic and then writes out the images to files:
class ImageConverter:
NUM_THREADS = 1
def __init__(self):
self.count = 0
self.todo = Queue.Queue()
self.stop = False
self.sub = None
self.threads = None
self.countLock = threading.Lock()
def __getCounter(self):
self.countLock.acquire()
c = self.count
self.count = self.count + 1
self.countLock.release()
ret = str(c)
while len(ret) < 4:
ret = "0" + ret
return ret
def __callback(self, data):
self.todo.put(data)
def __processImage(self,data):
bridge = CvBridge()
try:
source = bridge.imgmsg_to_cv(data, "32FC1")
except CvBridgeError, e:
print e
return
ret = cv.CreateImage((data.width, data.height), 8, 3)
mx = 0.0
for i in range(data.width):
for j in range(data.height):
d = source[j,i]
if d > mx:
mx = d
for i in range(source.width):
for j in range(source.height):
v = source[j,i]/mx*255
ret[j,i] = (v,v,v)
return ret
def startRecording(self,topic):
self.stop = False
def processor():
rate = rospy.Rate(100.0)
while (not self.stop) or (not self.todo.empty()):
if not self.todo.empty():
data = self.todo.get()
img = self.__processImage(data)
c = self.__getCounter()
cv.SaveImage("depth/frame" + c + ".jpg", img)
print("[ImageConverter]: image saved " + c + "/" + str(self.todo.qsize()))
rate.sleep()
self.threads = []
for i in range(ImageConverter.NUM_THREADS):
t = threading.Thread(target=processor, args = ())
t.daemon = True
t.start()
self.threads.append(t)
self.sub = rospy.Subscriber(topic, Image, self.__callback)
def stopRecording(self):
self.sub.unregister()
self.stop = True
for t in self.threads:
t.join()
As you can see in the __processImage method, I had to manually convert the image from 32FC1 to bgr8, since everytime I try to call the cv_bridge method imgmsg_to_cv with 'bgr8' (or 'mono8' for that matter) it throws an error saying it can't convert and saving directly as 32FC1 yields images that are thresholded to either max brightness or black. This is of course very slow, which is why I spin the processing off into a separate thread from the callback so I don't drop images.
After saving the series of images, I call mencoder to compile them into a video file, which usually reduces the total size down to about a fifth of what all the individual images take up:
subprocess.Popen("mencoder \"mf://*.jpg\" -mf type=jpg:fps=24 -o " + filename + " -speed 1 -ofps 24 -ovc lavc -lavcopts vcodec=mpeg2video:vbitrate=2500 -oac copy -of mpeg", cwd="temp/rgb/", shell=True, close_fds=True, stdout=DEVNULL, stderr=DEVNULL)
This seems like a pretty big hack. It seems like there should be a better way to do this, but after a few days of searching through forums and other ROS resources, I can't seem to find anything that can help. Additionally, the video files I get out of this are usually jumpy and sometimes frames are entirely missing. Is there any good way to record both RGB and Depth data as video files or a series of images using Python in ROS?
Originally posted by rse101 on ROS Answers with karma: 36 on 2014-02-22
Post score: 1