ROS + kinect depth data duplication

Question

I am trying to get depth data from a Kinect in a ROS project. It currently looks like this:

To arrive at this, I've done:

depth_sub = rospy.Subscriber("/camera/depth/image", Image, depth_cb)
...
def depth_cb(data):
    img = bridge.imgmsg_to_cv2(data, "32FC1")
    img = np.array(img, dtype=np.float32)
    img = cv2.normalize(img, img, 0, 1, cv2.NORM_MINMAX)
    cv2.imshow("Depth", img)
    cv2.waitKey(5)

I also launch openni.launch from the openni_launch package, which publishes the depth data.

I also get this weird warning from the node (can be seen in the image): ComplexWarning: Casting complex values to real discards the imaginary part.

But as I understand it the data type is an array of 32-bit floats. Yet some of the values appear as nan.

I would like a depth image that directly corresponds to a RGB image array of the same size. I will be doing some tracking in the RGB space, and using the tracked coordinates (X,Y) from that to index into the depth array. Thanks.

edit: Turns out, /camera/depth/image is published as an array of uint8s, but the actual data is 32bit floats (which is not listed anywhere, had to hunt it down in other people's code). Thus an array of 480x640 uint8s, interpreted as 32bit floats, in effectively "quartered" in the number of data points. Which could explain how the image is 4 times smaller (and hence accessing datapoints out of bounds = nan?), but not why there are two of them.

Answer 1

I hope I am understanding your issue right.

The "complex" error looks like an encoding issue: depth data does not contain any complex numbers. Try removing the CV_32FC1, let it fallback to "passthrough" mode, i.e., it will use whatever the default type is. OpenCV constructs its matrices in a weird way sometimes.

Are you using the Kinect V1 or V2? Neither Kinect has the same resolution on both color and depth camera, so can you tell me a little bit more about what issue you're having when you say "I would like a depth image that directly corresponds to a RGB image array of the same size"?