I am young researcher/developer coming from different (non-robotic) background and I did some research on camera localisation and I came to the point, where I can say that I am lost and I would need some of your help.

I have discovered that there is a lot of SLAM algorithms which are used for robots etc. As far as I know they are all in unknown environments. But my situation is different.

My problems and idea at the same time is:

  1. I will be placed in an known room/indoor environment (dimensions would be known)
  2. I would like to use handheld camera
  3. I can use predefined landmarks if they would help. In my case, I can put some " unique stickers" on the walls at predefined positions if that would help in any way for faster localisation.
  4. I would like to get my camera position (with its orientation etc) in realtime(30 Hz or faster).

For beginning I would like to ask which SLAM algorithm is the right one for my situation or where to start. Or do you have any other suggestions how to get real time camera positions inside of the known room/environment. It must be really fast and must allow fast camera movements. Camera would be on person and not on robot.

thank you in advance.

  • $\begingroup$ Oops just realized this question is from a long time ago. $\endgroup$ – Brian Lynch Dec 10 '15 at 5:14

As a matter of fact, if the environment is known a priori, then you don't need SLAM. It is a matter of localization problem which is simpler than SLAM in the sense that the map is given, therefore you don't need to estimate the map. In Probabilistic Robotics book chapter 7, the localization problem is addressed and they use probabilistic methods for the estimation process. They use 2D laser sensor. In your case, the heavy load is the vision part (i.e. extracting meaningful information from frames), other than that, you need an algorithm for the estimation process (e.g. EKF or Partical Filter) and data association algorithm for the process of matching between the observed features and the features in the given map (i.e. assuming your map is of feature-based type).

  • $\begingroup$ OK ill give some more detailed informations for my two projects that are related - at least I think they are. 1) ill have environment made of canvas in shape of cylinder (radious between 6-12m and 4m height) with some projections going on. Next person walks into this environment and holds in his hand wooden stick(camera). As he points to canvas with his stick my question/requested data is position + orientation of the end of the stick. I was thinking about IR camera on "stick" and IR led diods/stickers since projection will be changing and can confuce SLAM if I understand the basics. $\endgroup$ – Excalibur Aug 11 '15 at 17:37
  • $\begingroup$ Scenario 2) I would like to use pointcloud data from terrestrial laser scanner. For example I would scan my house from inside with laser scanner. This data would be as database/map for me. Next thing is that I would walk through my house with that stick(with camera) again and I would like to get same results in realtime as in my example 1). Is this even possible? $\endgroup$ – Excalibur Aug 11 '15 at 17:37

The problem you want to solve is definitely a SLAM problem and not just simply localization (or maybe we can consider it SLAm since the mapping part is not as heavy). The reason for this is that you will need to do some kind of mapping for initialization when you set up the environment. Of course you could set up the environment with markers or known features and carefully position them so that you know the map immediately, but that process is a lot harder than most people imagine.

Based on your question and the comments you've provided in CroCo's answer (which, by the way, is still a decent answer but I would like to expand), I see two paths that you can take which shouldn't be too difficult given all the resources available out there:

  1. Vision-based: Use fiduciary markers (QR codes or typical black and white checker patterns like you see on crash-test dummies), then locate these markers using computer vision feature recognition algorithms (OpenCV can do this easily using something like SURF). You can perform the initial mapping by using a graph-based SLAM approach (not real-time, record a bunch of data and crunch it all at once), then do localization only in real-time using a simplified bundle adjustment approach (with just one single image and enough visible features).

  2. LiDAR-based: Rely generally on ICP registration with your reference point cloud (pre-built map) and a fresh scan from the sensor. Note that to do this you need to use a scanner on your stick instead of a camera. It will be almost impossible to build the map with laser scan data and then localize inside it with a camera. I say "almost" because there does exist a body of research that relies on monocular cameras for generating 3D data from 2D images (shape from shading, for example), but that is going to be a much bigger challenge than I suspect you want to tackle.

Another interesting option is to use something like the X-Box Kinect (v1 or v2) so that you can get both 2D camera data and 3D scanning at the same time. With that approach you can then identify features in the camera images but have them linked to 3D coordinates in the scan data. Here is an example of the Kinect being used for indoor SLAM.

Some of the challenges you will encounter will be motion blur and even rolling shutter issues if you are using a fast-moving camera point-of-view. Your problem could be solved simply by considering a new image, identifying the features in that image, and then simple camera 3D reconstruction (again using OpenCV's camera reconstruction tools would be recommended).

I also suggest checking into using ROS, which provides a lot of the components for solving your problem already built and ready for implementation using a variety of different sensors.

Another thing that you haven't mentioned (and is alluded to in CroCo's answer) is that you can use an IMU (integrated accelerometer, gyroscope, and possibly magnetometer) to provide motion estimates of the camera itself. This is not sufficient alone but can make real-time pose estimation much easier (actually, most of us would consider it crucial). With the use of an IMU and camera/scanner data, you can then use an EKF or particle filter as has been suggested.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.