I am currently working on a project which which involves surround view.
I have 4 fish eye cameras and are fixed at 4 sides of a car.The fish eye cameras are corrected for radial distortion.
After radial correction,Each camera sees a pattern in the Common FOV of adjacent camera to get the points on the ground plane.
Pattern in the common FOV of adjacent camera
Now for each camera I need to warp those points to a plane which is the birds eye view.
Right now I choose those 8 red points, map first 4 points on a square and the other 4 points on another square on the same line,since I know that both squares lie parallel to each other at some distance,for front and back images and use the same points for left and right also appropriately so that left image comes left and right image comes right of the result.
Then I calculate Homography Matrix for each image(front,left,right and back) using the points in the Image and the birds eye plane.
I map the points such that warped front image sits at the top, left sits at the left side and right at the right side and then back sits at bottom of image .
Example, Front sits at the top of result image
I do this so that I can stitch properly,forming a composite view.
My final stitched image looks like below.
As you can see, the geometrical alignment is not proper.
What is the right way of registering each image with the ground plane.
From the object points shown as red dots, what is the proper way to get the corresponding points in the bird's eye plane.