Camera selection for long range stereo vision system (up to 100 meters)

Question

I want to implement a real-time stereo vision system for long range (up to 100m) depth estimation. I know that there are some general considerations as described in this SOV post. I have seen some typical cameras such as zed stereo cameras which has limited range (max. 20m).

Maximum allowable baseline distance between cameras is 0.5m and about field of view, i think that a camera with 8mm (or 12 or 16) focal length can provide reasonable FOV. I need depth resolustions at 100m to be 1% or maybe lower.

Are ip cameras proper for such applications? If no, Can anyone please suggest proper cameras for application in long range stereo vision system?

I will be grateful for any information you can provide.

Answer 1

I still think this is off-topic, but it seems I need more space than a comment to show (answer?) why that is so.

You are starting from some performance specifications and are looking to get to a set of features you need in your camera.

Here is a post from NI about stereo vision that gives a formula for depth resolution:

$$ \Delta z = \frac{z^2}{fb}\Delta d \\ $$

where $z$ is the depth of the object from the stereo system, $\Delta d$ is the depth resolution, $f$ is the focal length of the camera, $b$ is the baseline, and $d$ is the image disparity.

So, you want 1% depth resolution at 100 meters, or a depth resolution of 1 meter. A focal length of 8 millimeters, or 0.008 m, and a baseline of 0.5 m.

Rearranging the equation, it looks like you'll need a camera capable of registering a disparity of:

$$ \Delta d = \Delta z \frac{fb}{z^2} \\ \Delta d = 1 \frac{(0.008)(0.5)}{100^2} \\ \Delta d = 4 x 10^-7 m \\ \Delta d = 0.4 \mu m \\ $$

Assuming pixel accuracy (not sub-pixel accuracy), you'll want one pixel to be 0.4 $\mu$m or smaller, so the 0.4 $\mu$m disparity is registered as a one pixel shift between cameras.

Here's a list of sensor formats and sizes. I'm assuming these cameras all do "full HD", at a resolution of 1920x1080. Looking at the 2/3" format, the sensor width is 8.8 mm.

You need to register 0.4 $\mu$m, how does that compare to the 2/3" format? Well, at a width of 0.0088 m, with 1920 pixels across that width, the 2/3" format has a pixel width of $0.0088/1920 = 4.58\mu m$. So, off by a factor of 10. You need the pixel width to be about 11 times smaller.

So let's look at the 1/3" format - as in the iPhone 6. There the width is 4.8mm, so about half as wide, meaning you still need the pixels to be about 5-6 times smaller than the camera sensor in the iPhone.

This is also assuming you want to use every pixel in a full HD format - this will result in a high computation time. Most of the stereo vision projects I've seen have used cameras with lower resolutions or downsampled the image to a format like 640x480, but of course that means that the pixels are much (3x) larger.

You ask if "IP Cameras" are "proper," but IP cameras come in lots of styles.

Hopefully this will help you as a guide for your iterations. Plainly speaking, I don't think you'll ever find anything (that is reasonably affordable) that would do the depth resolution at the baseline you're talking about. I would imagine the baseline would be more on the range of 5-10 meters to get what you need. At 10 meters, fyi, the pixel size becomes 8 $\mu$m. At that point, most/all of the HD cameras should be able to do what you want, but again HD is computationally expensive because there are so many pixels to correlate.

This will be an iterative process. Work forward and backward and forward and backward until you get the design that meets your needs. You'll find you need to make tradeoffs along the way, and that's the core of engineering - finding the "optimal" balance of specifications. Cost, performance, size, cost, weight, interface, lead time, cost, cost, cost.

Answer 2

I'll try to explain this in terms of software. Firstly, it is just next to impossible to have 100 metres (practically, 3 metres is like the best depth accuracy) and we need to be sure that the cameras are capturing images properly with all practical features (vibration-free and even under sunlight or rain)! (Else features will be lost and we end up with almost 'plain' features resulting in poor mapping) While it has to deal largely with resolution, color, focal length etc, it is definitely practically impossible with normal IP camera and some specialised stereo-vision cameras may have to be employed as mentioned in Chuck's answer (0.4 micrometer?).

So, I would suggest the use of Stereo-vision or Laser scan for reinforcement learning (links not hyper-referenced deliberately).

1. "High Speed Obstacle Avoidance using Monocular Vision and Reinforcement Learning" http://www.cs.cornell.edu/~asaxena/rccar/icml_obstacleavoidance.pdf

2. "Depth Estimation using Monocular and Stereo Cues"
   https://pdfs.semanticscholar.org/4953/1103099c8d17ea34eb09433688e84de4f35f.pdf

These describe detecting features and associating depth-information with every feature and training it continuously for better and even better results. First paper uses LIDAR and that can be a bit costly for you. As 2D LIDAR is being used, the features of the image are sampled in columns and those features are mapped to image-kernels and the depth is thus estimated. In second case, stereo-vision is being used to train (note: this is high quality data as this is used for training) and as we have complete 3D information, features are being trained in a similar fashion as deep-nets and depth information is being obtained by comparing layers of information in image. These techniques can identify upto 50m or so effectively.

Also, I would suggest OP about asking a single question instead of asking duplicates like this.

Summary: It is practically next to impossible to get 100m depth with off-shelf cameras and no additional softwares.

Answer 3

old thread but the RPI hq camera with the 16mm lens would give you 1.875m at 100m with a 0.5m baseline. I think something that wasn't talked about much that I'm looking into is field of view and zoom and the affect on disparity.