# ROS + kinect depth data duplication

I am trying to get depth data from a Kinect in a ROS project. It currently looks like this:

To arrive at this, I've done:

depth_sub = rospy.Subscriber("/camera/depth/image", Image, depth_cb)
...
def depth_cb(data):
img = bridge.imgmsg_to_cv2(data, "32FC1")
img = np.array(img, dtype=np.float32)
img = cv2.normalize(img, img, 0, 1, cv2.NORM_MINMAX)
cv2.imshow("Depth", img)
cv2.waitKey(5)


I also launch openni.launch from the openni_launch package, which publishes the depth data.

I also get this weird warning from the node (can be seen in the image): ComplexWarning: Casting complex values to real discards the imaginary part.

But as I understand it the data type is an array of 32-bit floats. Yet some of the values appear as nan.

I would like a depth image that directly corresponds to a RGB image array of the same size. I will be doing some tracking in the RGB space, and using the tracked coordinates (X,Y) from that to index into the depth array. Thanks.

edit: Turns out, /camera/depth/image is published as an array of uint8s, but the actual data is 32bit floats (which is not listed anywhere, had to hunt it down in other people's code). Thus an array of 480x640 uint8s, interpreted as 32bit floats, in effectively "quartered" in the number of data points. Which could explain how the image is 4 times smaller (and hence accessing datapoints out of bounds = nan?), but not why there are two of them.

I hope I am understanding your issue right.

The "complex" error looks like an encoding issue: depth data does not contain any complex numbers. Try removing the CV_32FC1, let it fallback to "passthrough" mode, i.e., it will use whatever the default type is. OpenCV constructs its matrices in a weird way sometimes.

Are you using the Kinect V1 or V2? Neither Kinect has the same resolution on both color and depth camera, so can you tell me a little bit more about what issue you're having when you say "I would like a depth image that directly corresponds to a RGB image array of the same size"?

• I'll try removing the encoding - it's there because I'm following code samples :/ So what I mean is, I'm first doing some processing on a color image, which has resolution 480x640. Specifically, I'm tracking something in this image, so I get a (x,y) out of it. I thought the depth image is of the same size (480x640) and I can index into it do get the depth of that point, but according to what you just said, I can't do that because apparently they are NOT of the same resolution, something I probably would have benefitted from the beginning.
– Ben
Dec 4 '15 at 22:30
• Also using the v1 kinect.
– Ben
Dec 4 '15 at 22:32
• Why don't you downscale the RGB to 320x240 and go with it? Is that too low for your application? Dec 5 '15 at 3:38
• I have done basically that. I guess the crux of it is that I didn't realize the depth data would be at a different resolution.
– Ben
Dec 5 '15 at 10:14