In SLAM loop closure, What does a sim3 matrix do?

Question

I am confused between 3D rigid affine(with scale, without shear) transformation and Lie group's sim(3) matrix.

3D affine would be [sR|t; 0 1]. we rotate, scale and then translate a 3D point using 3D affine transform

\begin{matrix} s\cdot r_{11} & s\cdot r_{12} & s\cdot r_{13} & t_1\\ s\cdot r_{21} & s\cdot r_{22} & s\cdot r_{23} & t_2\\ s\cdot r_{31} & s\cdot r_{32} & s\cdot r_{33} & t_3\\ 0 & 0 & 0 & 1 \end{matrix}

sim3, according to [1] is [R|t; 0 1/s]

\begin{matrix} r_{11} & r_{12} & r_{13} & t_1\\ r_{21} & r_{22} & r_{23} & t_2\\ r_{31} & r_{32} & r_{33} & t_3\\ 0 & 0 & 0 & s^{-1} \end{matrix}

In loop closure context, for a candiate loop closure keyframe, we have an old pose(SE3) and its calculated sim3. sim3 has a scale, rotation and translation part.

• what is this sim3 actually? How do I get the new pose (SE3) using sim3 ?
• How do I get the new set of 3D points visible on the old keyframe pose?
  • In ORB slam code, the authors find the new set of 3D points from old set by first finding camera coordinates of the 3D points (using old pose) and then multiplying it by the inverse of calculated sim3
• If sim3 is a matrix that maps the old points to new points, why don't we multiply the 3D points directly with sim3 matrix?

Reference: [1] http://ethaneade.com/lie_groups.pdf

Answer 1

Sim3 is just the lie group associated a similarity transform which is the exact same thing as your 3D Affine transform. If you work out the multiplication for your two matrices you would find that they end up with the same result as we are dealing with homogenous coordinates. Therefore the following is true

$$x=kx$$

where $k$ is just some constant, and $x$ is a homogenous vector.

Now if we multiply a vector $[x,y,z,1]$ by your matrices. The results for the first row are shown below.

For affine matrix:

$$s(r_{11}*x + r_{12}*y + r_{13}*z) + t_1$$

Sim3 matrix:

$$r_{11}*x + r_{12}*y + r_{13}*z + \frac{t_1}{s^{-1}}$$ Multiply by $s$ $$s(r_{11}*x + r_{12}*y + r_{13}*z) + t_1$$

So the results end up differing by some scaling constant $s$, which means they are the same point.

what is this sim3 actually?

Similarity transform. Almost same thing as SE3, but with a extra degree of freedom for scaling. What you call 3D affine.

How do I get the new pose (SE3) using sim3?

You can't really. It has 1 extra degree of freedom so any conversion will lose information.

How do I get the new set of 3D points visible on the old keyframe pose?

You answered your own question. Multiply the point by the sim3 matrix. In ORB-SLAM they just break up the multiplication into the individual parts, but it is the same as multiplying by the matrix.

If sim3 is a matrix that maps the old points to new points, why don't we multiply the 3D points directly with sim3 matrix?

You can, and that's what ORB-SLAM even does. The inverse is just used cause the sim3 transform is in the wrong direction(e.g. from keyframe 2 -> keyframe 1) when you want the opposite.

In SLAM loop closure, What does a sim3 matrix do?

It does the same thing as the SE3 or SE2 transform in pose graph optimization. Add a constraint between poses. Sim3 is used instead as monocular based SLAM is special ,because scale/distance is unobservable. So the actual correct method to model this motion is sim3.

Answer 2

I think you'll find section IV. Loop Closure useful here: http://www.roboticsproceedings.org/rss06/p10.pdf