Understanding Drift in Simultaneous Localization and Mapping (SLAM)

I am trying to understand the effect of drift in Simultaneous Localization and Mapping (SLAM). My understanding is that drift occurs because the robot tracks its position relative to a set of landmarks it is storing, but each landmark has a small error in its location. Therefore, an accumulation of these small errors over a long trajectory causes a large error by the end of the trajectory.

However, what I am confused about is what would happen when the robot tracks its way back to its starting positions. Suppose the robot starts in position A, and then starts to move along a path, mapping the environment as it does so, until it reaches position B. Now, the robot will have some error in its stored position of B, due to the drift during tracking. But then suppose the robot makes its way back to A, by tracking relative to all the landmarks it created during the first path. When it reaches A, will it be back at the true position of A, i.e. where it started the first path? Or will it have drifted away from A?

My intuition is that it will end up at the true position of A, because even though the landmarks have errors in them, as long as the error is not too large then the robot will eventually get back to the position where it stored the landmarks for A. And once it is there, those landmarks are definitely correct, without error, because they were initialized before any drift errors had started to accumulate.

Any help? Thanks!

Once you've successfully closed the loop, depending on your SLAM algorithm, the uncertainty in your pose and the landmarks in your map becomes greatly reduced. This intuitively makes sense when you think about it. Let's say right before you close the loop, the uncertainty in your position was $\pm 3$ metres. Then your laser scanner (which has noise of $\pm 0.02$ metres) measures a landmark that you saw earlier, whose position you know pretty well (e.g., $\pm 0.05$ metres). It then makes sense that you know your position way better now! But wait, what about the other landmarks in your map? Their uncertainties are correlated with the uncertainty of your position because they were measured from the robot (these correlations manifest themselves in the off-diagonal terms of your covariance matrix). Then it only makes sense that drastically improving knowledge of your pose also improves your map!