I have a camera with a 1/2.9" image sensor from FLIR and a 12-50mm motorized zoom lens from Theia. Because the lens is a motorized zoom lens, I tried to calibrate the camera for different zoom and focus setups with an asymmetric circle matrix board and OpenCV calibrateCamera function. Each time I calibrate for a specific zoom and focus level, both zoom and focus are fixed.
Ideally, the principal point position (cx, cy) should be close to (720, 540) because the image resolution is 1440 x 1080. When the focal length is 12mm and 50mm, cx and cy look good, less than 60 pixels away from the center. However, they vary a few hundred pixels for some middle focal length settings, such as (834, 546), (913, 596), and (949, 613), but the image center seems to be stable when the lens is zooming and should not move for a few hundred pixels. The other camera intrinsic parameters also look good.
From the minimum to the maximum focal length, I observed that the distortion status changes from barrel distortion to pincushion distortion. From my understanding, there is a range that provides the minimum distortion. The status is also that the less distortion the image has, the worse the principal position is found.
Could anyone provide me any suggestions on solving this unstable principal point estimation problem?