I am currently busy with a final year project which requires me to track people walking through a doorway. I initially thought this may be possible using a normal camera and using some motion detection functions given in OpenCV, I have however come to the conclusion that the the camera is mounted too low for this to work effectively.(Height shown in the image below)

enter image description here

I have now been looking into using a 3D camera or a stereo camera to try and get around this problem.

I have seen similar examples where a Kinect(from Xbox 360) has been used to generate a depth map which is then processed and used to do the tracking, this was however done from a higher vantage point, and I found that the minimum operating range of the Kinect is 0.5m.

From what I have found, the Kinect uses an IR projector and receiver to generate its depth map, and have been looking at the Orbbec Astra S which uses a similar system and has a minimum working distance of 0.3m.

My question now:

What exactly would the difference be between the depth maps produced by a 3D camera that uses an IR projector and receiver, and a stereo camera such as the DUO/ZED type options?

I am just looking for some insight from people that may have used these types of cameras before

On a side note, am i going about this the right way? or should i be looking into Time of Flight Cameras instead?


My goal is to count the people moving into and out of the train doorway. I began this using OpenCV, initially with a background subtraction and blob detection method. This only worked for one person at a time and with a test video filmed at a higher vantage point as a "blob-merging" problem was encountered as shown in the left image below.

So the next method tested involved an optical flow method using motion vectors obtained from OpenCV's dense optical flow algorithm. From which i was able to obtain motion vectors from the higher test videos and track them as shown in the middle image below, because of the densely packed and easily detected motion vectors it was simple to cluster them.

But when this same system was attempted with footage taken from inside a train at a lower height, it was unable to give a consistant output. My thoughts of the reasons for this was because of the low height of the camera, single camera tracking is able to function when there is sufficient space between the camera and the top of the person. But as the distance is minimized, the area of the frame that the moving person takes up becomes larger and larger, and the space to which the person can be compared is reduced (or atleast that is how I understand it). Below on the right you can see how in the image the color of the persons clothing is almost uniform, Optical flow is therefore unable to detect it as motion in both cases.

enter image description here

I only started working with computer vision a few months ago so please forgive me if I have missed some crucial aspects.

From what i have seen from research, most commercial systems make used of a 3D cameras, stereo cameras or Time-of-Flight cameras, but I am unsure as to how the specifics of each of these would be best suited for my application.

  • $\begingroup$ What exactly is your goal? Do you want to identity people? If so, by what property? Or just count them? Given it's the latter, what's the problem with the image that you posted? I see a white blob of pixels and a dark blue one in front of a static scene. What's the problem detecting those? Should be possible with OpenCV, I'd say. $\endgroup$ Commented Jul 20, 2016 at 19:20
  • $\begingroup$ See above edit. Added history of use with OpenCV using single lens camera without success in desired system location. $\endgroup$ Commented Jul 20, 2016 at 22:00
  • $\begingroup$ Wow, great edit +1. My experience with computer vision is very limited, so I just leave another comment: my money would still be on the blob detection. I guess that it should be possible to keep the blobs separate by taking the likely different color of clothing into account. Depending on how many images you have, calculating the middle of the blob and filtering its movement (think kalman filter) could give enough information to separate blobs even if they merge in one image. $\endgroup$ Commented Jul 20, 2016 at 23:08

3 Answers 3


"3D camera" is a generalisation that covers sensors that can observe a point-cloud or depth map of the scene they are observing. Some 3D cameras use a projector (via various ways) to improve the resolution and/or reduce noise of these sensors.

The fact that the projector is IR is not important, it could be a red laser projector instead, but IR is used so that humans don't see the projection. Obviously if you use an IR projector you need an IR camera to observe the projection.

The projector will project a pattern that is either:

  • A texture (not necessarily a pattern) projected to the scene to give better features for stereo-vision by imposing texture on smooth areas (which is problematic for calculating stereo disparity).
  • A pattern that is coded, which is decoded to produces a depth map.
    • Various pattern types can be used (time-multiplexed, spatially-coded, direct-coded or a mixture)

A good example of both of these is the Intel Realsense F200 (coded-light) and the R200 (stereo vision).

Aside: You should start a separate question for your application, and narrow down the scope of your question.


This isn't an answer so much as a series of questions to get you to consider an appropriate solution.

I will, as usual :-), echo @BendingUnit22 and @Gouda - you appear to be fixated on a particular method of solution. This can be like, "What's a more fuel efficient way to buy groceries, a jet or a helicopter?" Well, can you just walk? If you go for a new sensor you'll need to learn to use it, which can be very time intensive. It's a color camera - can you split the image into R/G/B components and then do blob detection?

More importantly, consider your problem now: Two objects (people) get close together, and your blobs merge, and you can't tell them apart. What are you expecting is going to change when you get a depth component? The people still get close, and if they touch going through the door, then you now have a two-person-wide volume. This is exactly the same as the image blobs merging, except now you have an entire other dimension to work with.

If your primary issue is that you think the camera is mounted too low, why not just mount it higher? It would seem this would be dramatically simpler than purchasing and learning a new depth video system.

To answer your question specifically, any video system that gives depth should give video and depth. There will be differences in cost, resolution, accuracy, etc., but ultimately you should still get RGB+D. The Kinect uses structured light to get the depth map, stereo cameras use parallax, etc., but whatever the method to get the depth, the output is (generally) the same.

But again, to reiterate - consider what this system is actually going to get you and evaluate your current options again. There's almost certainly an easier way to do this.

  • 1
    $\begingroup$ I think the doorway that the camera is mounted to limits the height. Mounting it higher up is possible, but that means mounting it outside the door frame, which might reduce the applicability of the solution: For a building, the camera could be mounted higher up on the wall above the door, but it wouldn't be facing straight down or its view would be occluded partly by the wall. In transportation (like bus or tram), there's often not much above the doors to mount the camera to. A camera in the door frame seems to be a limiting but necessary requirement. $\endgroup$ Commented Jul 21, 2016 at 15:25
  • 1
    $\begingroup$ @BendingUnit22 That is correct, the images of in the train doorway in my original post are with the camera being mounted as high as i could possibly get it. I do see the point that is being made by Chuck though and will definitely spend some time looking into trying to solve the problem with what i currently have. $\endgroup$ Commented Jul 21, 2016 at 18:39

With stereo cameras, we are still dependent of having features/texture in the scene, regardless of using sparce or dense optical flow.

The depth is a function of the baseline, but we can have two cameras and control the baseline needed for the application.

So, I would say that the benefits are that we can select the resolution of the cameras, the frame rate and the field of view of the lens.

The system used by Kinect is widely available, for example in phones. Although is often not easy to access raw data.

But we can still do stereo with those systems if we have access to raw data. The stereo here is limited because typically the infrared camera has a lower resolution compared to the RGB camera and some systems have a wide angle RGB.

The advantage is that we have N-thousand points with depth that are in reflective surfaces and are not dependent of having texture.

Also, the IR depth is around 1m to 5m.

But I would use a fisheye in this application ;)

With this we would get early detecting and tracking.

The distortion would separate the heads from bodies, and it would facilitate separating people.

And we can get depth by measuring the motion with respect to the ground (planar induced homography).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.