Difference between 3D Camera(using IR projection) and Stereo Camera?

Question

I am currently busy with a final year project which requires me to track people walking through a doorway. I initially thought this may be possible using a normal camera and using some motion detection functions given in OpenCV, I have however come to the conclusion that the the camera is mounted too low for this to work effectively.(Height shown in the image below)

I have now been looking into using a 3D camera or a stereo camera to try and get around this problem.

I have seen similar examples where a Kinect(from Xbox 360) has been used to generate a depth map which is then processed and used to do the tracking, this was however done from a higher vantage point, and I found that the minimum operating range of the Kinect is 0.5m.

From what I have found, the Kinect uses an IR projector and receiver to generate its depth map, and have been looking at the Orbbec Astra S which uses a similar system and has a minimum working distance of 0.3m.

My question now:

What exactly would the difference be between the depth maps produced by a 3D camera that uses an IR projector and receiver, and a stereo camera such as the DUO/ZED type options?

I am just looking for some insight from people that may have used these types of cameras before

On a side note, am i going about this the right way? or should i be looking into Time of Flight Cameras instead?

----EDIT----:

My goal is to count the people moving into and out of the train doorway. I began this using OpenCV, initially with a background subtraction and blob detection method. This only worked for one person at a time and with a test video filmed at a higher vantage point as a "blob-merging" problem was encountered as shown in the left image below.

So the next method tested involved an optical flow method using motion vectors obtained from OpenCV's dense optical flow algorithm. From which i was able to obtain motion vectors from the higher test videos and track them as shown in the middle image below, because of the densely packed and easily detected motion vectors it was simple to cluster them.

But when this same system was attempted with footage taken from inside a train at a lower height, it was unable to give a consistant output. My thoughts of the reasons for this was because of the low height of the camera, single camera tracking is able to function when there is sufficient space between the camera and the top of the person. But as the distance is minimized, the area of the frame that the moving person takes up becomes larger and larger, and the space to which the person can be compared is reduced (or atleast that is how I understand it). Below on the right you can see how in the image the color of the persons clothing is almost uniform, Optical flow is therefore unable to detect it as motion in both cases.

I only started working with computer vision a few months ago so please forgive me if I have missed some crucial aspects.

From what i have seen from research, most commercial systems make used of a 3D cameras, stereo cameras or Time-of-Flight cameras, but I am unsure as to how the specifics of each of these would be best suited for my application.

Answer 1

"3D camera" is a generalisation that covers sensors that can observe a point-cloud or depth map of the scene they are observing. Some 3D cameras use a projector (via various ways) to improve the resolution and/or reduce noise of these sensors.

The fact that the projector is IR is not important, it could be a red laser projector instead, but IR is used so that humans don't see the projection. Obviously if you use an IR projector you need an IR camera to observe the projection.

The projector will project a pattern that is either:

• A texture (not necessarily a pattern) projected to the scene to give better features for stereo-vision by imposing texture on smooth areas (which is problematic for calculating stereo disparity).
• A pattern that is coded, which is decoded to produces a depth map.
  • Various pattern types can be used (time-multiplexed, spatially-coded, direct-coded or a mixture)

A good example of both of these is the Intel Realsense F200 (coded-light) and the R200 (stereo vision).

Aside: You should start a separate question for your application, and narrow down the scope of your question.

Answer 2

This isn't an answer so much as a series of questions to get you to consider an appropriate solution.

I will, as usual :-), echo @BendingUnit22 and @Gouda - you appear to be fixated on a particular method of solution. This can be like, "What's a more fuel efficient way to buy groceries, a jet or a helicopter?" Well, can you just walk? If you go for a new sensor you'll need to learn to use it, which can be very time intensive. It's a color camera - can you split the image into R/G/B components and then do blob detection?

More importantly, consider your problem now: Two objects (people) get close together, and your blobs merge, and you can't tell them apart. What are you expecting is going to change when you get a depth component? The people still get close, and if they touch going through the door, then you now have a two-person-wide volume. This is exactly the same as the image blobs merging, except now you have an entire other dimension to work with.

If your primary issue is that you think the camera is mounted too low, why not just mount it higher? It would seem this would be dramatically simpler than purchasing and learning a new depth video system.

To answer your question specifically, any video system that gives depth should give video and depth. There will be differences in cost, resolution, accuracy, etc., but ultimately you should still get RGB+D. The Kinect uses structured light to get the depth map, stereo cameras use parallax, etc., but whatever the method to get the depth, the output is (generally) the same.

But again, to reiterate - consider what this system is actually going to get you and evaluate your current options again. There's almost certainly an easier way to do this.

Answer 3

With stereo cameras, we are still dependent of having features/texture in the scene, regardless of using sparce or dense optical flow.

The depth is a function of the baseline, but we can have two cameras and control the baseline needed for the application.

So, I would say that the benefits are that we can select the resolution of the cameras, the frame rate and the field of view of the lens.

The system used by Kinect is widely available, for example in phones. Although is often not easy to access raw data.

But we can still do stereo with those systems if we have access to raw data. The stereo here is limited because typically the infrared camera has a lower resolution compared to the RGB camera and some systems have a wide angle RGB.

The advantage is that we have N-thousand points with depth that are in reflective surfaces and are not dependent of having texture.

Also, the IR depth is around 1m to 5m.

But I would use a fisheye in this application ;)

With this we would get early detecting and tracking.

The distortion would separate the heads from bodies, and it would facilitate separating people.

And we can get depth by measuring the motion with respect to the ground (planar induced homography).