# why the left 3x3 portion of Projection matrix(P) is different from the Intrinsic camera matrix(K)?

Intrinsic camera matrix for the raw
(distorted) images. [fx 0 cx]
K = [ 0 fy cy]
[ 0 0 1]
Projects 3D points in the camera coordinate frame to 2D pixel coordinates using the focal lengths (fx, fy) and principal point (cx, cy).

Projection/camera matrix
[fx' 0 cx' Tx]
P = [ 0 fy' cy' Ty]
[ 0 0 1 0]
By convention, this matrix specifies the intrinsic (camera) matrix of the processed (rectified) image. That is, the left 3x3 portion is the normal camera intrinsic matrix for the rectified image. It projects 3D points in the camera coordinate frame to 2D pixel coordinates using the focal lengths (fx', fy') and principal point (cx', cy') - these may differ from the values in K.

inage width: 1920 image height:1080 canera nane: narrow stereo
cameramatrix: rows: 3 cols:3
data:[1963.58626 0. 978.96792 0. 1961.4452 548.41778 0. 0. 1.]
distortion_model: plumb bob
distortion coefficients: rOWS: 1 cols:5
data: [-0.554099, 0.343450,-0.000465. 0.001727, 0.000000]
rectification matrix: rows: 3 cols:3
data: [1. 0. 0. 0. 1. 0. 0.0. 1. ]
projection matrix rows. 3 cols:4
data: [1679.60999 0. 989.73115 0. 0.1875.7949 548.96323 0. 0. 0. 1. 0. ]

when i use the camera_calibration cameracalibrator.py to calibrate the camera, the result shows the left 3x3 portion of Projection matrix(P) is different from the Intrinsic camera matrix(K), why they are different?

how to caculate from Intrinsic camera matrix(K) to Projection matrix(P)?

Originally posted by koughua on ROS Answers with karma: 13 on 2021-12-16

Post score: 1

Have a look at this page, which is also linked in the sensor_msgs/CameraInfo comments: https://wiki.ros.org/image_pipeline/CameraInfo

Specifically this part:

4.1 Simple monocular rectification

In this case, there is a monocular device with distortion correction only. To transform a pixel in the input image into one in the output image, it is sent through the K - D - R - K' series of transformations. K - D gets to the normalized, undistorted image; the rotation R is the identity because we don't want to rotate the normalized undistorted image; and then K' converts back to pixel coordinates in the output image. In this case, since there is no need of in-image-plane translation, or scaling from the original image, so K = K' , and only the K and D elements of CameraInfo are needed. But since ROS Electric, the camera_calibration package does monocular calibration to get K' using OpenCV's getOptimalNewCameraMatrix() function with argument 'alpha'=0.0 that cause K' ≠ K.

Projection onto the output image is by the P matrix, which is formed from K' and the optional rotation and translation, in this case the identity and 0, respectively. So if K' = K, P is redundant information, and could be formed directly from K.

The important point here is that the left 3x3 portion of P is computed by getOptimalNewCameraMatrix() function with argument 'alpha'=0.0. If you undistort an image, it is no longer rectangular, and you'll get a pincushioning effect like this:

Since the image has to be rectangular to be stored in memory, the remaining pixels are filled with a black color.

The getOptimalNewCameraMatrix() scales the camera matrix K and computes the new image size so that the undistorted (rectified) image either contains no invalid pixels (alpha = 0.0), or so that all original pixels are contained in the image (alpha = 1.0) or something in between. Also, the new image size for the rectified image has to be specified. The camera_calibration package uses alpha = 0 (i.e., crop the image) and output image size = input image size.

And if you're working with stereo camera pairs, it gets even more complicated (see the page I linked above), so in that case P can never be replaced with just using K.

Short rule of thumb: If you're working with rectified images, use P. If you're working with unrectified (raw) images, use K.

Originally posted by Martin Günther with karma: 11816 on 2021-12-16

This answer was ACCEPTED on the original site

Post score: 1