4
$\begingroup$

I am currently working on a project which which involves surround view.

I have 4 fish eye cameras and are fixed at 4 sides of a car.The fish eye cameras are corrected for radial distortion.

After radial correction,Each camera sees a pattern in the Common FOV of adjacent camera to get the points on the ground plane.

Pattern in the common FOV of adjacent camera

enter image description here

Now for each camera I need to warp those points to a plane which is the birds eye view.

Right now I choose those 8 red points, map first 4 points on a square and the other 4 points on another square on the same line,since I know that both squares lie parallel to each other at some distance,for front and back images and use the same points for left and right also appropriately so that left image comes left and right image comes right of the result.

Then I calculate Homography Matrix for each image(front,left,right and back) using the points in the Image and the birds eye plane.

I map the points such that warped front image sits at the top, left sits at the left side and right at the right side and then back sits at bottom of image .

Example, Front sits at the top of result image

enter image description here

Left sits at the Left of result imageenter image description here

I do this so that I can stitch properly,forming a composite view.

My final stitched image looks like below.

enter image description here

As you can see, the geometrical alignment is not proper.

QUESTIONS

  1. What is the right way of registering each image with the ground plane.

  2. From the object points shown as red dots, what is the proper way to get the corresponding points in the bird's eye plane.

Improve this question
$\endgroup$
3
  • $\begingroup$ How do you stitch the images together ? $\endgroup$
    – Malcolm
    Commented Aug 13, 2017 at 2:31
  • $\begingroup$ @Malcolm I start from the front image and then go clockwise and replace the older overlapped part with the current image. $\endgroup$
    – Nitron_707
    Commented Aug 13, 2017 at 14:15
  • $\begingroup$ you can try big chessboard pattern which covers whole area, then do extrinsic calibration. This may give high accuracy $\endgroup$
    – user18675
    Commented Nov 16, 2017 at 6:06

1 Answer 1

Reset to default
2
$\begingroup$

I don't know what the proper way to do this is but I can give you ideas to make it better:

  • define an exact transformation for each camera. You could calibrate each camera using a grid pattern and by using a bigger one, multiple pictures, and averaging the transforms, you could define the transformation between the fish eye view and the normal view more accurately and wouldn't need the pattern on the image later on. Im on my phone so I can find any more info or links on this but there are plenty :).

  • use image registration techniques. Use keypoints and descriptors on the image and register then using something like ransac for example. A good library to do that is openCV. Look at this for example but I think you can find plenty of better tutorials.

Ideally, you should use both of those technics :).

Improve this answer
$\endgroup$
1
  • $\begingroup$ Have tried many things...but without the actual destination points from image plane to bird's eye view plane, it is really tough..else we need an Ideal image taken exactly from the top(what I am doing now) so that we get a reference image for for bird eye view plane..then we can construct a proper homography between points in the image and the corresponding points in the IDEAL reference image....I do follow your techniques to obtain homography matrix...Thank you for ur time $\endgroup$
    – Nitron_707
    Commented Aug 14, 2017 at 14:59

Not the answer you're looking for? Browse other questions tagged or ask your own question.