Mobile Camera Calibration and rectification frame rate

Question

I've been searching the internet for an answer to this question, but I haven't come across anything that will help me. Basically, we have a Meka Humanoid Robot in our lab, which has a shell head in which a PointGrey USB 3.0 camera is embedded. I use the pointgrey_camera_driver for obtaining the images from the camera. The head has 2 degrees of freedom (up/down, left/right). I am intending to use this camera with the ar_pose package to detect and track AR tags on objects. I understand that camera's must be calibrated for effective use (forgive me, I don't know much about camera) which I do with the camera_calibration package.

My question is: Since this camera is "mobile" meaning since the head can move so does the camera, how would I go about calibrating it? Currently, I have the head fixed at a position and I've calibrated the camera in that position and got the parameters in the yaml file which I can load for rectification. In particular, if the head moves does the calibration file that I obtained in the previous position become invalid? If so, as asked before, how would I calibrate this camera for all of its possible field's of view (which can be obtained by moving)?

This camera has different video modes and in the mode I'm using I can get a frame rate of 21Hz (i.e. after driver is launched I get 21Hz for rostopic hz /camera/image_raw). However, when I rectify the image using image_proc, I get a frame rate of only about 3Hz on rostopic hz /camera/image_rect_color. Is this normal? Is there a way for me to increase this frame rate?

Please let me know if any more information is required. Thanks for your help!

Answer 1

When you refer to "camera calibration" I assume you mean the estimation of the camera/lens principal point, focal lengths and distortion coefficient; these are called the intrinsic parameters. From a calibration you can also determine the extrinsic parameters (camera position and orientation) at the time the image was taken. From what I can understand you do not need the extrinsics.

Your questions:

how would I calibrate this camera for all of its possible field's of view (which can be obtained by moving)?

if the head moves does the calibration file that I obtained in the previous position become invalid?

If your camera and lens are constant-zoom and not auto-focus (as many Point Grey research cameras are, unsure what lens you're using) then you can simply calibrate the camera once and you are done. The intrinsics will not change and remain valid.

I always calibrate using the MATLAB CALTECH camera calibration toolbox, however there are less time-consuming automated software such as Agisoft Lens (I can't find the official website for some reason). As a standard I usually calibrate using at least 21 images (the 21 is from wisdom from a lecturer). When you get the values it is very important to save ALL the values as you do not want to recalibrate twice, one calibration may take up to an hour if you are quick.

If your camera is not constant-zoom or is auto-focus then your camera intrinsics change almost constantly, I have never tried to work with such a camera due to its difficulty.

EDIT: When the "field of view" (please read definition) changes the focal length or zoom has changed, which will invalidate the intrinsics acquired from camera calibration. Simply changing the pose of the camera will not invalidate the intrinsic parameters acquired from calibration.