This is the basic slam problem. You have to find out (model) how the uncertainty of the robot affects the uncertainty of the landmarks, and visa-versa. This is done using the cross-correlation terms of the uncertainty. Or, the covariance. Those four things you referred to as covariances are actually variances. The covariance describes how the undertainties in one thing bleed over to create uncertainty in the other things. This sounds terrible, but actually, if you reduce uncertainty in one thing, then the fact that two things co-vary allows you to reduce uncertainty in two things.
See, when you discuss covariance of the robot's position, what you mean is something like a 2x2 matrix. The matrix has multiple terms, two variances in the principal x-y directions, and the covariance of the x uncertainty and the y uncertainty. So you're already doing SLAM. Sorta.
What you have to do for $N$ lamps in $2D$ space and 1 robot moving in $2D$, is construct the giant $2\times (N+1)$ by $2\times (N+1)$ matrix. How to do this is mechanically straightforward, but an intuitive understanding is difficult to build up. I'm sure someone will come along that has a better mechanical answer for you. But until then, you might check out Probabilistic Robotics, by Thrun.