Correct Fish eye camera radial distortion and blurry stretch at the image edges

Question

There are a lot of questions regarding this topic, but I am trying to get a more clear picture from these questions.

I am trying to calibrate a fish eye camera and I am using OpenCV cv::omnidir class functions to find the camera intrinsics.

I am getting fair results. The problem is, at the image edges, the objects get stretched and I am, also, losing some information at the edges.

Here is my input image:

Here is my output image:

As you can see, I am losing some information at the edges (left and right) and also the images start stretching at the edges.

My questions are as follows:

1. How I can I include more FOV in the corrected image at the edges, where the information is lost?
2. How can I reduce the blur effect at the edges?
3. During calibration, should I cover the entire FOV of the camera so that the corners are present at the edges also?
4. What is the correct way of showing the patterns while calibration?
5. Are there any online tool boxes which provides fish eye calibration.

Here is my code snippet for calibration and testing:

//Calibration
Mat K, xi, D, idx;
int flags=0|omnidir::CALIB_FIX_SKEW | omnidir::CALIB_FIX_K1 | 
 omnidir::CALIB_FIX_K2;

TermCriteria critia(cv::TermCriteria::COUNT + cv::TermCriteria::EPS, 200, 
0.0001);

vector<cv::Mat> rvecs, tvecs;

double rms = cv::omnidir::calibrate(obj_points, image_points, image_size, K, 
xi, D, rvecs, tvecs, flags, critia, idx);


//Testing
Mat R = Mat::eye(3, 3, CV_32F);
Mat Mapx, Mapy;
Mat New_camera_mat(3,3,CV_32F);

//New_camera_mat tries to get entire FOV,but it is losing some information 
  at edges

New_camera_mat.at<float>(0, 0) = 100; New_camera_mat.at<float>(0, 1) = 0; 
New_camera_mat.at<float>(0, 2) = 1280/2;

New_camera_mat.at<float>(1, 0) = 0; New_camera_mat.at<float>(1, 1) = 100; 
New_camera_mat.at<float>(1, 2) = 720/2 ;

New_camera_mat.at<float>(2, 0) = 0; New_camera_mat.at<float>(2, 1) = 0; 
New_camera_mat.at<float>(2, 2) = 1;

cv::omnidir::initUndistortRectifyMap(K, D, xi_Right, R, New_camera_mat, 
image_size, CV_32F, Mapx, Mapy, cv::omnidir::RECTIFY_PERSPECTIVE);

remap(distorted_frame, undistorted_out_frame, Mapx, Mapy, INTER_CUBIC);

Answer 1

How I can I include more FOV in the corrected image at the edges, where the information is lost?

-> Just try to modify fx and fy of your intrinsic matrix. Maybe, multiplying a scalar to K(0,0) and K(1,1) will work.

How can I reduce the blur effect at the edges?

-> Generally, it is not possible unless you create a specialized DNN. All the blurry area is because you are stretching small areas in the original image. It is an inevitable side effect when you undistort a fish-eye image.

During calibration, should I cover the entire FOV of the camera so that the corners are present at the edges also? -> Yes

What is the correct way of showing the patterns while calibration? Are there any online tool boxes which provides fish eye calibration.

-> To fully observe the lens distortion characteristic you should show the pattern everywhere in the image. Try to show your board around the circular border of the image. But not too much as blurry corners will decrease the accuracy.

-> No open source online one as far as I know. If need one you should build one.