# Converting Pixel Coordinates to spatial coordinates: Python, Ros

Hi,

So currently, i am reading an image from my robot's camera, located on its hand. I want to obtain the set of coordinates of a certain object, from both the camera's perspective and the robot's base perspective, given a certain pixel coordinate from the camera i.e. 340x200 (location of my object on the camera image)

I have been using the equation on the second line from this link

But i haven't obtained any good results.(I have inverted the equation,, since i want a set of 3d coordinates from a pair of pixel coordinates and not the opposite as the original equation would give)

So because i didnt get any good result, i tried breaking down the equation. The matrix [R|T] converts the [x,y,z,1] coordinates to the base frame right?? (assuming [R|T] is with respect to the base frame)

So if I wanted the coordinate WRT to the camera's frame, could I just use the equation, without the [R|T] matrix? i.e [u,v,1] = A[X,Y,Z]?

If not, how should i proceed to convert the pixel coordinates to spatial coordinates

Originally posted by laurent on ROS Answers with karma: 21 on 2016-06-17

Post score: 0

I didn't really read the details of your question very closely, nor did I look at the link. It seems to me that there are so many different conventions and coordinate systems used for camera geometry it often leads to confusion. If you are using a camera calibrated with camera_calibration, and the camera information is available as a CameraInfo topic, then I'd recommend using the image_geometry package. The two packages are specifically designed to work together, and the image_geometry package contains a number of useful functions. Check out the Python API and the C++ API.

Originally posted by jarvisschultz with karma: 9031 on 2016-06-17

This answer was ACCEPTED on the original site

Post score: 1

Comment by laurent on 2016-06-20:
I tried using image_geometry, but i get a 'NoneType' object has no attribute 'getitem' error when using it. Here's what i did

self.img_geo = PinholeCameraModel()
(r_x,r_y,r_z) = self.img_geo.projectPixelTo3dRay((359,28))


any idea whats going on?

Comment by laurent on 2016-06-20:
I obviously imported the packages beforehand

Comment by jarvisschultz on 2016-06-20:
You need to read in the camera calibration parameters

cam_info = rospy.wait_for_message("/camera1/camera_info", CameraInfo, timeout=None)
img_proc = PinholeCameraModel()
img_proc.fromCameraInfo(cam_info)
n = img_proc.projectPixelTo3dRay((30,30))


Comment by laurent on 2016-06-20:
So I added that bit to my code, but it seems that the code stops at rospy.wait_for_message and doesnt compute the rest

Comment by laurent on 2016-06-20:
So I figrued that that cam_info was basically just subscribing to a camera, so i replcaed the wait_for_message with directly subscribing to the camera. and the error that i get says that 'Subscriber' object has no attribute 'K'

Comment by jarvisschultz on 2016-06-20:
Your first comment indicates, most likely, that you weren't publishing a CameraInfo message on the /camera1/camera_info topic. Your second comment indicates that you are passing the created subscriber into the fromCameraInfo method. That is not correct.

Comment by jarvisschultz on 2016-06-20:
The wait_for_message function creates a subscriber, waits until the topic is published, and returns a copy of the message that was received. My code doesn't return a subscriber, it returns a CameraInfo message that has been filled out with information from camera calibration.

Comment by laurent on 2016-06-20:
So given that I revert back to wait_for_message, how do i publish a CameraInfo message? You can't publish to the camera right, you can only subscribe to it

Comment by jarvisschultz on 2016-06-20:
Most camera drivers provide functionality to automatically publish this topic based on a calibrated camera yaml file. You should also be able to manually publish this from the command line using rostopic pub.

Comment by laurent on 2016-06-20:
It turns out i was wrongly entering my topic. I managed to get it to work, thank you so much, your help has been immensely useul

Comment by jarvisschultz on 2016-06-20: