How can I remap where ar_pose gets camera_info?

Question

Hello all, I'm trying to use ar_pose with a fisheye lens on a GigE Vision camera (specifically, a Vimba). Since the lens doesn't expose the whole sensor I am left with a choice, either use cameracalibration.py with very poor x coverage, or set the region of interest in the camera to only send back the square of pixels that actually contain information and calibrate on that. I tried it the first way first with mediocre results so I'm trying it the second way now but I'm hitting a problem; when I hit Store to store the calibration parameters in the camera it errors out because the resolution of the calibration doesn't match the resolution of the sensor.

What I want to do is use the camera with the reduced region of interest, but change the node that ar_pose gets the camera_info topic from (currently it's from the driver that gets the info from the camera, I want to replace it with something that reads it from disk). I don't think that's too hard to do, I'm just a ROS newb.

Edit: Here are the two calibrations;

Poor X coverage:

header: 
  seq: 911
  stamp: 
    secs: 1461698059
    nsecs: 40779009
  frame_id: camera
height: 1458
width: 1936
distortion_model: plumb_bob
D: [-0.15703, 0.014130000000000002, 6.000000000000001e-05, 0.0025, 0.0]
K: [482.37535, 0.0, 985.71393, 0.0, 478.79862, 728.41403, 0.0, 0.0, 1.0]
R: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]
P: [1043.20752, 0.0, 935.81685, 0.0, 0.0, 1110.39636, 727.20482, 0.0, 0.0, 0.0, 1.0, 0.0]
binning_x: 1
binning_y: 1
roi: 
  x_offset: 300
  y_offset: 50
  height: 1358
  width: 1358
  do_rectify: True
---

Good coverage, cannot save into the camera:

# oST version 5.0 parameters
   
[image]

width
1358

height
1358

[narrow_stereo]

camera matrix
470.978444 0.000000 651.245530
0.000000 454.436211 686.264331
0.000000 0.000000 1.000000

distortion
-0.153380 0.013363 0.000422 -0.000092 0.000000

rectification
1.000000 0.000000 0.000000
0.000000 1.000000 0.000000
0.000000 0.000000 1.000000

projection
343.130035 0.000000 629.437552 0.000000
0.000000 325.993042 680.842927 0.000000
0.000000 0.000000 1.000000 0.000000

turtlebot@vision1:~/catkin_ws/src/massdest_bringup/calibrations$ cat ost.yaml 
image_width: 1358
image_height: 1358
camera_name: narrow_stereo
camera_matrix:
  rows: 3
  cols: 3
  data: [470.978444, 0.000000, 651.245530, 0.000000, 454.436211, 686.264331, 0.000000, 0.000000, 1.000000]
distortion_model: plumb_bob
distortion_coefficients:
  rows: 1
  cols: 5
  data: [-0.153380, 0.013363, 0.000422, -0.000092, 0.000000]
rectification_matrix:
  rows: 3
  cols: 3
  data: [1.000000, 0.000000, 0.000000, 0.000000, 1.000000, 0.000000, 0.000000, 0.000000, 1.000000]
projection_matrix:
  rows: 3
  cols: 4
  data: [343.130035, 0.000000, 629.437552, 0.000000, 0.000000, 325.993042, 680.842927, 0.000000, 0.000000, 0.000000, 1.000000, 0.000000]

---

Originally posted by dunmatt on ROS Answers with karma: 1 on 2016-04-25

Post score: 0

---

Original comments

Comment by lucasw on 2016-04-26:
what was the x y offset to the roi you set in the camera?

Comment by lucasw on 2016-04-26:
Your poor x coverage camera matrix numbers look better- mainly because fx and fy are closer to each other (for most cameras and lenses fx=fy, but the sensor + lens + calibration error are going to make them a little different). The distortion number seem sufficiently similar between calibrations.

Answer 1

I've done a lot of camera calibration, though not much with cameracalibration.py, so these are more educated guesses rather than definite solutions:

You should save the calibration to disk, uncompress the tar.gz file then have the camera driver load ost.yaml from disk (and move it out of /tmp) when it starts rather that using the set_camera_info service. Does your camera driver have a camera_info_url parameter?

As long as you can still generate and save the calibration it shouldn't matter if the x coverage gets a low score- just make it as high as you can. (If camera calibrator showed reprojection error that is what you would be concerned with- the ros industrial camera calibration may be better at that http://rosindustrial.org/news/2016/1/29/new-intrinsic-calibration-procedure, I haven't used it though)

But if you want to use the roi calibration numbers you would need to modify them in the yaml file- add the roi x and y offset to the cx cy camera and projection matrix parameters, and fix the resolution to be the full sensor size. Once that is done it would be interesting to compare the adjusted roi calibration to the low x coverage full image calibration- do they vary a lot? (copy and paste the numbers into your question)

Fisheye calibration with the standard plumb_bob model probably isn't going to give you great results. You could jump through extra hoops to use opencv 3.0 fisheye calibration, but then image_proc and other standard tools likely including ar_pose won't be able to use the calibration. (Maybe newer versions compiled from source can? Otherwise maybe you could find or write a node to fishey rectify the image, and then pass that to ar_pose)

---

Originally posted by lucasw with karma: 8729 on 2016-04-25

This answer was ACCEPTED on the original site

Post score: 0

---

Original comments

Comment by dunmatt on 2016-04-26:
I'm afraid the driver doesn't seem to have a way to specify the camera_info. Isn't there a way to use ROS to force ar_pose to get that topic from a different node than the driver?

Or alternatively, are the two distortion models I added above close enough that it doesn't matter?

Comment by lucasw on 2016-04-26:
Some of my answer to http://answers.ros.org/question/232931/zoom-camera-calibration-in-ros/ might help you- you can remap the one camera_info and bring in the other one from elsewhere, as long as the timestamps match.

Comment by lucasw on 2016-04-26:
Also your camera driver ought to be able to load your camera yaml from disk, so you wouldn't need inject a different camera_info.

Comment by lucasw on 2016-04-26:
If this is your driver look at https://github.com/srv/avt_vimba_camera/blob/indigo/launch/mono_camera.launch#L10