In my understanding, in order to improve accuracy for the estimation of depth with a stereo vision camera, patterns of light are projected into the scene, because the resulting texture makes it easier to find point correpondances between the pair of stereo images, thus we can triangulate correct pairs of points and produce a dense and accurate depth map. But as far as I know, we can't see this structured light in the images captured by the stereo camera (nor in the scene), so how does a stereo camera make use of structured light in stereo matching?
1
-
$\begingroup$ if you think about it, the camera has to be able to see the pattern, otherwise the system would not function correctly ... you can probably see the pattern if the illumination spectrum is within the human eye sensitivity and if the illumination pattern persistence is long enough $\endgroup$– jsotolaJul 7, 2019 at 21:48
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
The majority of sensors using structured light actually use the deformation of the light to compute the 3D depth directly. They do not use it to augment/improve another depth estimation technique, such as stereo parallax. By projecting a known pattern the distortions of the pattern can be converted directly into 3D depth information (after matching the features of the structure, not image pixels). Typically distortions due to the sensor (e.g. lens) are accounted for by calibrating against known surfaces. Many systems use multiple cameras so that the structure light can be seen even if obstructed from particular angles (and to provide additional sensing).
If you are referring to sensors that use structured light only to improve on depth information of a technique such as stereo imaging, they may work differently (I'm unaware of any).
$\begingroup$
$\endgroup$
@DavidJ is referring to sensors like the Kinect1, Asus Xtion, Intel SR300 which project a pattern of IR dots and observe the deformation with a monocular camera over a baseline.
Stereo cameras like the Intel D415 use stereo vision plus an IR pattern projector. Stereo algorithms match patterns of pixels between the two views but if the surface is very plain, ie. a white wall, the matching fails. The IR patter projector provides an artificial texture which allows the stereo to succeed. With the RealSense SDK you can turn the projector on/off and see the difference in stereo performance. The projector has a limited range but works well indoors where ambient IR from the sun is not a problem and where textureless surfaces are prevalent. Outdoors, the projector is less effective (it has to compete with the sun) but the world is generally much more textured.
