1
$\begingroup$

it is widely discussed in this forum, how to sync multiple cameras (e.g. best done via hardware trigger). Now I'd like to know how to actually determine the time delay between two image streams and the overall offset between a real event, and the moment when the captured frame is available in the PCs buffer.

  1. So I have a picture P which captured an event at time T, so the captured image will be available at P(T+dT). How to determine dT?

  2. I have two cameras capturing picture P1 and P2. But due to two unsynchronized shutter, P_1 will be captured at T_1 and P_2 will be captured at T_2. How to determine dt=T2-T1?

PS.: I really don't want to synchronize the cameras, just determining these delays. I searched intuitivley for publications on multi camera/Kinect tracking or Event Related Synchronisation but weren't lucky at all. Does someone has some suggestions?

Improve this question
$\endgroup$
9
  • $\begingroup$ A word of caution: Many CMOS image sensors have what is called a "rolling shutter" which captures scan lines at different times (I believe to reduce noise due to current surges). As a result, even a single image will represent a progression in time. This may be a difficult question to answer in a non camera-specific way. $\endgroup$
    – Tut
    Sep 19, 2017 at 18:28
  • 2
    $\begingroup$ A simple approach would be to turn the lights off then on, and then measure the delay between when each camera notices the off and on. This will work if the delay is a constant - otherwise I know there are neural network approaches which have done pretty well at compensating for frame delay, but that takes quite a bit of work. $\endgroup$
    – combo
    Sep 19, 2017 at 18:42
  • $\begingroup$ @combo This looks like a heavy approach, due to DNNs. But I am looking for a more analytical approach. In the end, I'll extract features, so my approach is to do linear interpolation between two consecutive frames to get the same timestamps among cameras. But for this, I need to determine the lag. Further, I also thought about a light on/off (using flash). But there has to be some literature regarding this issue, or is it that uncommon?Rolling shutter is really an issue, but the impact of movement is neglected for our first approach. $\endgroup$
    – Tik0
    Sep 20, 2017 at 8:34
  • $\begingroup$ I think you are going to have to dive deep into how the image is created and handled. From the basics of shifting the image off the CCD surface (2009 Physics Nobel prize stuff) to either avoiding or understanding image compression. If you are searching further, the only people I know who desire camera to camera synchronization are 3D movie makers. But I do not think they care about latency like you do. Lastly, if this is a controlled setting (like a lab) simply place a no-latency time dependent display (NOT an LCD, use LEDs) in the field of view to indicate the T the frame was captured. $\endgroup$
    – st2000
    Sep 21, 2017 at 11:45
  • $\begingroup$ "multi camera/Kinect tracking", I think you are right, the Kinect uses multiple cameras. But I do not think there is any effort spent to discover the latency between the cameras. I believe Kinect uses constructive light and an infrared (likely a black & white) camera to determine depth (object placement). I do not see any latency issues there to be concerned about. And, far as I understand, the color camera is not even used for 3D tracking. (Maybe the color camera is used for applications like Skype.) So no latency issues that I can think of there either. $\endgroup$
    – st2000
    Sep 21, 2017 at 13:11

1 Answer 1

Reset to default
1
$\begingroup$

Consider using infrared transmitters to flood the area of interest with short bursts of infrared light while using video cameras sensitive to infrared light or both infrared and visible light.

By flooding the area of interest with detectable pulses, the video from each camera can be analyzed such that any time discrepancies due to unexpected latencies can be mitigated.

The location of objects in three dimensional space can be improved as video from various angles can be combined without concern regarding when in time a particular image from a particular camera was taken.

Improve this answer
$\endgroup$
3
  • $\begingroup$ This sounds really like a feasible approach I'll try. But just to be sure: Is a method based on intuition, or do you have some references where someone already has tried this? $\endgroup$
    – Tik0
    Sep 28, 2017 at 23:14
  • $\begingroup$ I thought I had read the description a Wikipedia article. And it follows what I have noticed. That commercial multiple camera 3d capture labs are outfitted with big infrared arrays around each camera. $\endgroup$
    – st2000
    Sep 29, 2017 at 3:21
  • $\begingroup$ I think what you refer to is the VICON system, am I right? This is, of course, a completely different level and modality (from the point of money and technology) of tracking in comparison what I try to achieve with commercial cameras. $\endgroup$
    – Tik0
    Sep 29, 2017 at 20:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.