3
$\begingroup$

In context of the paper Semi Dense Visual Odometry for a Monocular Camera J Engel, J Sturms, D Cremers

At page 7, last paragraph the author writes

The achieved tracking accuracy for two feasible sequences - that is, sequences which do not contain strong rotations without simultaneous translation - is given in Table 1.

Here the Author implies that the method is not feasible (fails ?) on datasets that contain strong rotation. In the paper, Visual Odometry is performed in two parts inverse depth estimation of points and tracking of camera. I want to analyze how these parts are affected when the camera undergoes strong rotations?

Here is what I think (kindly correct me and or add missing details)

Part I
Inverse depth estimation is based on disparity among two successive frames. The observed pixel disparity is equal to scaled inverse depth for the pixel.

When camera undergoes strong rotation it causes apparent pixel disparity however the pixel would still be at the same depth wrt the camera. So this results in false disparity and subsequently wrong depth estimation (?)

Part II
The tracking of camera is being done via image alignment (using depth map). The image alignment is formulated as a photometric error minimization problem:

$$ E(\xi) := \sum_{i}\frac{\alpha(r_i(\xi))}{\sigma_{d_i}^2}r_i(\xi)$$ where $$r_i(\xi) = (I_2(~{\color{blue}w(x_i, d_i, \xi)}~) - I_1(x_i))^2$$ and ${\color{blue}w}$ is a warp function that maps each point $x_i$ in reference image $I_1$ to the respective point in the new image $I_2$

The warp function requires depth $d_i$ to calculate the new position of a pixel in image $I_2$ that was observed earlier in $I_1$. With wrong depth estimate the minimization problem would yield wrong value for $\xi$ (?)

Also, are there techniques to deal with pure camera rotation in case of Monocular Vision based Odometry?

Improve this question
$\endgroup$
3
  • $\begingroup$ The depth perception with a monocular image in the way the authors did it can only work if there is more than one image of the same scene, from slightly different viewpoints (as would be the case for binocular vision - the two simultaneous images of binocular vision are replaced by two concurrent images at slight offset of camera position, off course betting on the mostly unchanging scene between two or more successive pictures). A pure rotation is not delivering this changed view point of the scene, and can thus not be used for depth perception $\endgroup$
    – bukwyrm
    Commented Aug 5, 2019 at 8:20
  • $\begingroup$ @bukwyrm Thanks for the answer. Intuitively, what you say is the basis for temporal stereo with monocular camera. As you can see in Part I the disparity (which also results from the offset of camera position) is still there in the observed scene in presence of a rotation. In that case the cause for the failure of the method is not apparent (disparity due to rotation should produce wrong depth estimate). And, in Part II, it appears to me that the image alignment problem should work fine given only rotations, but would it really work? I am trying to get a deeper insight. $\endgroup$
    – vyi
    Commented Aug 6, 2019 at 2:31
  • $\begingroup$ Sorry, i thought you did not get the technique, when in truth i did not get your explication.. Still, i don't get what you miss: The author says they can't work with strong rotations that do not at the same time contain translations - that is pure, strong rotations. In case of pure rotations, any depth map that is produced has no basis in reality, because it is exclusively made up from rotation-induced errors. There simply is no temporal stereo with no translation. And without a good, frame-filling depthmap, your rot&trans estimation on the next frame will be worthless. $\endgroup$
    – bukwyrm
    Commented Aug 6, 2019 at 5:27

1 Answer 1

Reset to default
1
$\begingroup$

The problem of the pure strong rotation is that the image will become easily blurred unless you use 1000FPS camera. This kind of super-strong rotation often occurs in hand-held camera motion. Simply turn on your mobile camera in a slightly dark place and try to make a translational and rotational motion. You will see rotation make huge blurriness in the image where tracking or depth estimation will surely fail.

Improve this answer
$\endgroup$
5
  • $\begingroup$ I see, so blurring is one reason for tracking failure. Are there other sources of failure. If we have a hypothetical camera that is able to capture images without blur at 25fps. Would temporal stereo (as in paper) be able to correctly track and estimate the depth of semi dense points (under strong rotations)? $\endgroup$
    – vyi
    Commented Aug 8, 2019 at 15:48
  • $\begingroup$ Yes, I am sure of that but it should be faster than 25fps. This is the reason why commercial solutions are using a camera with a global shutter and high fps(over 60). Rolling shutter also makes problem with fast rotation. $\endgroup$
    – Chanoh Park
    Commented Aug 9, 2019 at 1:17
  • $\begingroup$ Many of the past slam research was done with low-cost webcams. A camera with high fps and global shutter used to be very expensive. $\endgroup$
    – Chanoh Park
    Commented Aug 9, 2019 at 1:20
  • $\begingroup$ That makes sense. I tried to use lsd-slam with a sequence captured via smartphone camera and I am getting aweful result (method loses tracking). Also, I notice the authors claim the method to be fast but when I run it, the tracking itakes time of order of 500ms or more (on a i5 processor 8th generation)! I do not have access to global shutter camera. $\endgroup$
    – vyi
    Commented Aug 9, 2019 at 1:27
  • $\begingroup$ I thank you for your valuable inputs. I would like to know your comments about the inferences I have made in the question (part I and part II). That would make the answer more complete ! $\endgroup$
    – vyi
    Commented Aug 9, 2019 at 1:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.