Why do strong rotations affect Monocular Vision based Visual Odometry? - Robotics Stack Exchange most recent 30 from robotics.stackexchange.com 2019-12-14T15:40:08Z https://robotics.stackexchange.com/feeds/question/19218 https://creativecommons.org/licenses/by-sa/4.0/rdf https://robotics.stackexchange.com/q/19218 2 Why do strong rotations affect Monocular Vision based Visual Odometry? vvy https://robotics.stackexchange.com/users/23497 2019-08-05T08:06:21Z 2019-08-08T05:55:56Z <p>In context of the paper <a href="http://openaccess.thecvf.com/content_iccv_2013/papers/Engel_Semi-dense_Visual_Odometry_2013_ICCV_paper.pdf" rel="nofollow noreferrer">Semi Dense Visual Odometry for a Monocular Camera</a><sup> J Engel, J Sturms, D Cremers</sup></p> <p>At page 7, last paragraph the author writes </p> <blockquote> <p>The achieved tracking accuracy for two feasible sequences - that is, sequences which do not contain strong rotations without simultaneous translation - is given in Table 1.</p> </blockquote> <p>Here the Author implies that the method is not feasible (fails ?) on datasets that contain strong rotation. In the paper, Visual Odometry is performed in two parts <strong>inverse depth estimation</strong> of points and <strong>tracking</strong> of camera. I want to analyze how these parts are affected when the camera undergoes <strong>strong rotations</strong>? </p> <p>Here is what I think (kindly correct me and or add missing details) </p> <p>Part <strong>I</strong><br> Inverse depth estimation is based on disparity among two successive frames. The observed pixel disparity is equal to scaled inverse depth for the pixel.</p> <p>When camera undergoes <strong>strong rotation</strong> it causes apparent pixel disparity however the pixel would still be at the same depth wrt the camera. So this results in false disparity and subsequently wrong depth estimation (?) </p> <p>Part <strong>II</strong><br> The tracking of camera is being done via image alignment (using depth map). The image alignment is formulated as a photometric error minimization problem:</p> <p><span class="math-container">$$E(\xi) := \sum_{i}\frac{\alpha(r_i(\xi))}{\sigma_{d_i}^2}r_i(\xi)$$</span> where <span class="math-container">$$r_i(\xi) = (I_2(~{\color{blue}w(x_i, d_i, \xi)}~) - I_1(x_i))^2$$</span> and <span class="math-container">${\color{blue}w}$</span> is a warp function that maps each point <span class="math-container">$x_i$</span> in reference image <span class="math-container">$I_1$</span> to the respective point in the new image <span class="math-container">$I_2$</span> </p> <p>The warp function requires depth <span class="math-container">$d_i$</span> to calculate the new position of a pixel in image <span class="math-container">$I_2$</span> that was observed earlier in <span class="math-container">$I_1$</span>. With wrong depth estimate the minimization problem would yield wrong value for <span class="math-container">$\xi$</span> (?)</p> <p>Also, are there techniques to deal with pure camera rotation in case of Monocular Vision based Odometry?</p> https://robotics.stackexchange.com/questions/19218/-/19239#19239 1 Answer by C.O Park for Why do strong rotations affect Monocular Vision based Visual Odometry? C.O Park https://robotics.stackexchange.com/users/19219 2019-08-08T05:55:56Z 2019-08-08T05:55:56Z <p>The problem of the pure strong rotation is that the image will become easily blurred unless you use 1000FPS camera. This kind of super-strong rotation often occurs in hand-held camera motion. Simply turn on your mobile camera in a slightly dark place and try to make a translational and rotational motion. You will see rotation make huge blurriness in the image where tracking or depth estimation will surely fail. </p>