# Camera selection for long range stereo vision system (up to 100 meters)

I want to implement a real-time stereo vision system for long range (up to 100m) depth estimation. I know that there are some general considerations as described in this SOV post. I have seen some typical cameras such as zed stereo cameras which has limited range (max. 20m).

Maximum allowable baseline distance between cameras is 0.5m and about field of view, i think that a camera with 8mm (or 12 or 16) focal length can provide reasonable FOV. I need depth resolustions at 100m to be 1% or maybe lower.

Are ip cameras proper for such applications? If no, Can anyone please suggest proper cameras for application in long range stereo vision system?

I will be grateful for any information you can provide.

• Welcome to Robotics, ObjectOrientedMan. Questions like this ("What X do I need for Y?") are generally off-topic because they are shopping questions. Any products we list here (especially digital cameras) are unlikely to be relevant or even sold as time progresses, so the question will be of little use to future visitors. The better question is, "How can I determine if a particular camera is suitable for my application?" That question is relevant for future visitors, because it explains what characteristics to look for. – Chuck Mar 10 '17 at 22:02
• The post to which you linked does a fair job at going over how particular characteristics are related to stereo system performance. You ask for a recommended camera for a particular distance, but what are your limitations and requirements? This is what will drive the camera selection. For instance, what is your maximum allowable baseline distance between cameras? What is your depth resolution requirement? What is your required field of view? Even if it were on-topic, you need to nail down what you want before anyone can give recommendations. – Chuck Mar 10 '17 at 22:06
• I'm closing the question (off-topic as described above), but if you'd like to discuss your problem, please join us in Robotics Chat. If you have a question about how to determine a particular performance specification, then those are the kinds of questions we would love to see! – Chuck Mar 10 '17 at 22:07
• Thank you @Chuck . I have edited question considering your critique. I just have asked a general question about cameras for long range stereo vision system not for providing a specific product. A question is opinion-based when there are lots of available answers, but I don't think that it would be true for question. – Mr. Nobody Mar 11 '17 at 3:02
• I humbled with much information I get from above Q/A. In offshore structure installation there is a real need for such stereoscopic cameras to be utilized for monitoring the installation of structures to ensure that positioned on seabed within pre-determined tolences of Absolute position: +/- 0.5m Absolute heading: +/- 2.5° inclination in pitch / roll: +/- 0.2° The range between sensor and structure could reach up to 25m. The baseline between two cameras could be 10 or 15m. What matters here, not to create point cloud / image with high resolution but to aid the installation of the structure wi – Ghoneim Dec 2 '17 at 1:19

I still think this is off-topic, but it seems I need more space than a comment to show (answer?) why that is so.

You are starting from some performance specifications and are looking to get to a set of features you need in your camera.

Here is a post from NI about stereo vision that gives a formula for depth resolution:

$$\Delta z = \frac{z^2}{fb}\Delta d \\$$

where $z$ is the depth of the object from the stereo system, $\Delta d$ is the depth resolution, $f$ is the focal length of the camera, $b$ is the baseline, and $d$ is the image disparity.

So, you want 1% depth resolution at 100 meters, or a depth resolution of 1 meter. A focal length of 8 millimeters, or 0.008 m, and a baseline of 0.5 m.

Rearranging the equation, it looks like you'll need a camera capable of registering a disparity of:

$$\Delta d = \Delta z \frac{fb}{z^2} \\ \Delta d = 1 \frac{(0.008)(0.5)}{100^2} \\ \Delta d = 4 x 10^-7 m \\ \Delta d = 0.4 \mu m \\$$

Assuming pixel accuracy (not sub-pixel accuracy), you'll want one pixel to be 0.4 $\mu$m or smaller, so the 0.4 $\mu$m disparity is registered as a one pixel shift between cameras.

Here's a list of sensor formats and sizes. I'm assuming these cameras all do "full HD", at a resolution of 1920x1080. Looking at the 2/3" format, the sensor width is 8.8 mm.

You need to register 0.4 $\mu$m, how does that compare to the 2/3" format? Well, at a width of 0.0088 m, with 1920 pixels across that width, the 2/3" format has a pixel width of $0.0088/1920 = 4.58\mu m$. So, off by a factor of 10. You need the pixel width to be about 11 times smaller.

So let's look at the 1/3" format - as in the iPhone 6. There the width is 4.8mm, so about half as wide, meaning you still need the pixels to be about 5-6 times smaller than the camera sensor in the iPhone.

This is also assuming you want to use every pixel in a full HD format - this will result in a high computation time. Most of the stereo vision projects I've seen have used cameras with lower resolutions or downsampled the image to a format like 640x480, but of course that means that the pixels are much (3x) larger.

You ask if "IP Cameras" are "proper," but IP cameras come in lots of styles.

Hopefully this will help you as a guide for your iterations. Plainly speaking, I don't think you'll ever find anything (that is reasonably affordable) that would do the depth resolution at the baseline you're talking about. I would imagine the baseline would be more on the range of 5-10 meters to get what you need. At 10 meters, fyi, the pixel size becomes 8 $\mu$m. At that point, most/all of the HD cameras should be able to do what you want, but again HD is computationally expensive because there are so many pixels to correlate.

This will be an iterative process. Work forward and backward and forward and backward until you get the design that meets your needs. You'll find you need to make tradeoffs along the way, and that's the core of engineering - finding the "optimal" balance of specifications. Cost, performance, size, cost, weight, interface, lead time, cost, cost, cost.

• +1 I upvoted because the efforts are nice although the question could have been clearer with OP asking about his actual project requirements and why. – Prasad Raghavendra Mar 13 '17 at 15:08
• Thank you @Chuck. Excuse me for the late reply. I thought that the post is closed. I have a question about your answer. Why you have assumed one pixel accuracy? I think that if I use algorithms with 1/10 sub-pixel matching accuracy, Δd becomes a reasonable value. In this link (link.springer.com/chapter/10.1007/978-3-319-10605-2_7), authors have shown that with sub-pixel matching accuracy (0.25 to 1/20), we can achieve reasonable errors at long ranges. Can I overcome the hardware limitations with implementing optimized algorithms with sub-pixel matching accuracy? Again thank for your answer. – Mr. Nobody Mar 15 '17 at 19:25
• @ObjectOrientedMan - I took a look at the paper; note that they state in at section 4.1, "We consider disparities between 9 and 3 pixels." I'm no stereo vision expert, but I don't know that you can do sub-pixel disparity mapping if you don't have at least some pixel-level disparity. – Chuck Mar 15 '17 at 21:15

I'll try to explain this in terms of software. Firstly, it is just next to impossible to have 100 metres (practically, 3 metres is like the best depth accuracy) and we need to be sure that the cameras are capturing images properly with all practical features (vibration-free and even under sunlight or rain)! (Else features will be lost and we end up with almost 'plain' features resulting in poor mapping) While it has to deal largely with resolution, color, focal length etc, it is definitely practically impossible with normal IP camera and some specialised stereo-vision cameras may have to be employed as mentioned in Chuck's answer (0.4 micrometer?).

So, I would suggest the use of Stereo-vision or Laser scan for reinforcement learning (links not hyper-referenced deliberately).

1. "High Speed Obstacle Avoidance using Monocular Vision and Reinforcement Learning" http://www.cs.cornell.edu/~asaxena/rccar/icml_obstacleavoidance.pdf

2. "Depth Estimation using Monocular and Stereo Cues"
https://pdfs.semanticscholar.org/4953/1103099c8d17ea34eb09433688e84de4f35f.pdf

These describe detecting features and associating depth-information with every feature and training it continuously for better and even better results. First paper uses LIDAR and that can be a bit costly for you. As 2D LIDAR is being used, the features of the image are sampled in columns and those features are mapped to image-kernels and the depth is thus estimated. In second case, stereo-vision is being used to train (note: this is high quality data as this is used for training) and as we have complete 3D information, features are being trained in a similar fashion as deep-nets and depth information is being obtained by comparing layers of information in image. These techniques can identify upto 50m or so effectively.