Zoom/scale image combined with rectify/undistort and publish modified CameraInfo in a ros nodelet?

Question

I'm running the rectify nodelet on a factory calibrated oak-d-w camera and I don't like how much of the original distorted image gets chopped off as a result. I think I saw there was an alternate rectification node or nodelet that can apply a scale factor and do the right modifications to the camera info and publishes it out, but now I'm not finding it.

Curved black border areas are fine as consequence for scaling the image too much (though it would be nice if it could optionally automatically calculate the right scale factor to be right on the edge of having undefined black pixels in the output image, or make sure every input pixel contributes to the output image- using getOptimalNewCameraMatrix()?).

Answer 1

So far I'm able to get most of what I want by adding blank borders to the distorted input image and offsetting cx & cy in the corresponding camera info (and making the camera info width and height match the larger image size).

I added a Pad nodelet to image_manip that does this https://github.com/lucasw/image_manip/blob/master/image_manip/src/pad.cpp#L69-L88 (the pad parameters need to be made dynamic reconfigure variables, or at least rosparams). Maybe the rectification nodelet could take padding parameters and would be much more efficient without having to modify the input distorted image?

Really wide angle cameras (like the oak-d-w) need a lot of padding to get more of the edges of the image, it's diminishing returns and increasingly odd looking.

(I'll add some helpful images here when I have a good example to share)