Having a camera mounted on my robot and looking upwards, I want to estimate the distance of the ceiling as the robot moves and also the position of landmarks observed on the ceiling (lamps for example). I know this is a structure from motion problem but I was very confused on how to implement it. This case is a much simpler case than bundle adjustment as the intrinsic calibration of the camera is existing, the camera pose changes just in x and y directions, and the observed thing is a planar ceiling. Odometry might also be available but I would like to start solving it without. Do you know any libraries that offer a good and simple API to do such a thing? preferably based on levenberg-marquardt or similar optimization algorithms taking in more than just two observations. (Python bindings would be nice to have)
-
$\begingroup$ "Odometry might also be available but I would like to start solving it without. " That won't be possible as you only have a single camera so that your reconstruction is only defined up to scale. This scale could be defined by observing an object with known size or by using the absolute scale of you odometry (if it's good enough). You can implement your problem with ceres or g2o, both are often used for this type of task to that there are already good examples for error-definitions using reprojection errors. $\endgroup$– FooTheBarSep 2, 2015 at 16:53
-
$\begingroup$ From my understanding, in bundle adjustment similar problems there is no information about relative pose of the cameras, not even the intrinsic parameters of the camera and a reconstruction in real size 3D is possible. So what is the difference in my case? Is it the fact that I am observing a planar surface? $\endgroup$– MehdiSep 3, 2015 at 8:23
-
$\begingroup$ Your reconstruction will always only be defined up to an arbitrary scale. Just google "estimating scale in structure from motion" and you will find many approaches how to estimate this scale. (cs.nyu.edu/~fergus/teaching/vision_2012/6_Multiview_SfM.pdf p.38) The scale of the scene and the focal length of your camera have the same effect on the reprojection error. $\endgroup$– FooTheBarSep 3, 2015 at 8:42
-
$\begingroup$ what you say is that having multiple pictures from different positions delivers exactly the same amount of information as having one picture from one position (I could also define real world coordinates X and Y of observed points up to a certain assumed scale, the depth Z), of course when working with planar surfaces. $\endgroup$– MehdiSep 3, 2015 at 8:59
-
$\begingroup$ You will get a 3d model, but you don't know the scale of the scene. If you could compute the scale, you could also do the intrinsic calibration of your camera without knowing the size of the pattern, right? But you need the scale as you can't distinguish a large patten that is far away from a smaller pattern that is closer to your camera. $\endgroup$– FooTheBarSep 3, 2015 at 9:07
2 Answers
Not sure if I understood the problem correctly, but I understood that you wish to estimate the height of the observed objects hanging from the ceiling.
You have a mono camera but you can take two images from different positions and use them as a stereo pair. You can use OpenCV to do this more easily: http://docs.opencv.org/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html
Most packages utilize stereo images to calculate distances. StereoVision is a python package that can be used to generate 3d point clouds. Also, this will require the use of odometry information.
In order to utilize information from more than two sequential images, an implementation of extended kalman filter can be utilized. Successive point clouds can be used to update the estimate of the ceiling.
