# How to calculate observation matrix Jacobians in EFK based mono slam? What are projection Jacobians?

I have been trying to understand an implementation of monoSLAM 1. But my question is generic to EKF based single camera SLAM.

My Jacobian calculation leads to complex and lengthy expressions while the c++ implementation [3] seems to have done it very easily with what they call 'projection Jacobian' and its inverse. I don't understand the math involved and why they are calculated. Please point out where my understanding is wrong.

The following is my current understanding of the subject of VSLAM using EKF.

In monoslam, the 3D landmarks are measured indirectly using image coordinates as we do not get the depth information from a single camera. They are either parametrized as 3D points $$L$$ ($$L_x, L_y, L_z$$) or as 6D inverse depth formulation $$L_{inv}$$ ($$C_x, C_y, C_z, \theta, \phi, \rho$$). In EKF, the non-linear function $$h()$$ that converts the state vector into image measurements of landmarks involves the 3D to 2D projection equations (Camera model). The formula for converting a 3D point or a inverse depth formulation involves camera position, orientation, camera parameters and landmark representations $$L$$ or $$L_{inv}$$.

I have difficulty in calculating the Jacobians of the $$h()$$ function as part of the update equations. I tried calculating the partial differentiation of $$h()$$ with respect to the different elements of my state vector (used matlab symbolic tool) and got lengthy Jacobians.

The SLAM paper uses the projection model and calculates the Jacobians as mentioned in the section 2.5 of [2]. With respect to what the partial differentiation is done? To my understanding they have calculated w.r.t the image coordinates and not the elements from the state vector.

Hope my write up is clear.

References:

1. Russo, Ludovico, et al. "A ROS implementation of the mono-slam algorithm" International Journal of Computer Science & Information Technology 6.1 (2014): 339-351
2. https://github.com/rrg-polito/mono-slam/blob/master/reference/Master%20Thesis%20Ludovico%20Russo%20-%20oneside.pdf

3. https://github.com/rrg-polito/mono-slam

• It then struck me that the authors have used chain rule. But it would be great if some one can point me to a document that explains these Jacobians in detail to understand how the chain rule has been applied. Commented Feb 17, 2019 at 23:25