# In SLAM loop closure, What does a sim3 matrix do?

I am confused between 3D rigid affine(with scale, without shear) transformation and Lie group's sim(3) matrix.

3D affine would be [sR|t; 0 1]. we rotate, scale and then translate a 3D point using 3D affine transform

$$\begin{matrix} s\cdot r_{11} & s\cdot r_{12} & s\cdot r_{13} & t_1\\ s\cdot r_{21} & s\cdot r_{22} & s\cdot r_{23} & t_2\\ s\cdot r_{31} & s\cdot r_{32} & s\cdot r_{33} & t_3\\ 0 & 0 & 0 & 1 \end{matrix}$$

sim3, according to [1] is [R|t; 0 1/s]

$$\begin{matrix} r_{11} & r_{12} & r_{13} & t_1\\ r_{21} & r_{22} & r_{23} & t_2\\ r_{31} & r_{32} & r_{33} & t_3\\ 0 & 0 & 0 & s^{-1} \end{matrix}$$

In loop closure context, for a candiate loop closure keyframe, we have an old pose(SE3) and its calculated sim3. sim3 has a scale, rotation and translation part.

• what is this sim3 actually? How do I get the new pose (SE3) using sim3 ?
• How do I get the new set of 3D points visible on the old keyframe pose?
• In ORB slam code, the authors find the new set of 3D points from old set by first finding camera coordinates of the 3D points (using old pose) and then multiplying it by the inverse of calculated sim3
• If sim3 is a matrix that maps the old points to new points, why don't we multiply the 3D points directly with sim3 matrix?

Sim3 is just the lie group associated a similarity transform which is the exact same thing as your 3D Affine transform. If you work out the multiplication for your two matrices you would find that they end up with the same result as we are dealing with homogenous coordinates. Therefore the following is true

$$x=kx$$

where $$k$$ is just some constant, and $$x$$ is a homogenous vector.

Now if we multiply a vector $$[x,y,z,1]$$ by your matrices. The results for the first row are shown below.

For affine matrix:

$$s(r_{11}*x + r_{12}*y + r_{13}*z) + t_1$$

Sim3 matrix:

$$r_{11}*x + r_{12}*y + r_{13}*z + \frac{t_1}{s^{-1}}$$ Multiply by $$s$$ $$s(r_{11}*x + r_{12}*y + r_{13}*z) + t_1$$

So the results end up differing by some scaling constant $$s$$, which means they are the same point.

what is this sim3 actually?

Similarity transform. Almost same thing as SE3, but with a extra degree of freedom for scaling. What you call 3D affine.

How do I get the new pose (SE3) using sim3?

You can't really. It has 1 extra degree of freedom so any conversion will lose information.

How do I get the new set of 3D points visible on the old keyframe pose?

You answered your own question. Multiply the point by the sim3 matrix. In ORB-SLAM they just break up the multiplication into the individual parts, but it is the same as multiplying by the matrix.

If sim3 is a matrix that maps the old points to new points, why don't we multiply the 3D points directly with sim3 matrix?

You can, and that's what ORB-SLAM even does. The inverse is just used cause the sim3 transform is in the wrong direction(e.g. from keyframe 2 -> keyframe 1) when you want the opposite.

In SLAM loop closure, What does a sim3 matrix do?

It does the same thing as the SE3 or SE2 transform in pose graph optimization. Add a constraint between poses. Sim3 is used instead as monocular based SLAM is special ,because scale/distance is unobservable. So the actual correct method to model this motion is sim3.

• Apologize for not being able to comment. I have a question about @edwinem's answer. The result of the first row after multiplying the vector with the Sim3 matrix should be $$r_{11}*x + r_{12}*y + r_{13}*z + t_1$$ There is no $s$ in the first row. If we multiply the vector $X = [p; 1]$, where $p=[x, y, z]$, by the rigid affine transformation, the result is $$[sRp+t; 1].$$ Meanwhile, the multiplication between X and the Sim(3) matrix is $$[R*p+t; s^{-1}] \approx [s*R*p+s*t; 1]$$ The difference between the two results above is not by scale. Did I miss anything? Commented Oct 6, 2021 at 3:23

I think you'll find section IV. Loop Closure useful here: http://www.roboticsproceedings.org/rss06/p10.pdf

• While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. Commented Jun 28, 2020 at 16:23