# Image registration with ground plane for surround view

I am currently working on a project which which involves surround view.

I have 4 fish eye cameras and are fixed at 4 sides of a car.The fish eye cameras are corrected for radial distortion.

After radial correction,Each camera sees a pattern in the Common FOV of adjacent camera to get the points on the ground plane.

Pattern in the common FOV of adjacent camera

Now for each camera I need to warp those points to a plane which is the birds eye view.

Right now I choose those 8 red points, map first 4 points on a square and the other 4 points on another square on the same line,since I know that both squares lie parallel to each other at some distance,for front and back images and use the same points for left and right also appropriately so that left image comes left and right image comes right of the result.

Then I calculate Homography Matrix for each image(front,left,right and back) using the points in the Image and the birds eye plane.

I map the points such that warped front image sits at the top, left sits at the left side and right at the right side and then back sits at bottom of image .

Example, Front sits at the top of result image

Left sits at the Left of result image

I do this so that I can stitch properly,forming a composite view.

My final stitched image looks like below.

As you can see, the geometrical alignment is not proper.

QUESTIONS

1. What is the right way of registering each image with the ground plane.

2. From the object points shown as red dots, what is the proper way to get the corresponding points in the bird's eye plane.

• How do you stitch the images together ? Aug 13, 2017 at 2:31
• @Malcolm I start from the front image and then go clockwise and replace the older overlapped part with the current image. Aug 13, 2017 at 14:15
• you can try big chessboard pattern which covers whole area, then do extrinsic calibration. This may give high accuracy Nov 16, 2017 at 6:06