0
$\begingroup$

Assuming you have a fixed stereo camera, looking at a rigid object, what are the techniques used to track the object?

My google searches lead me to optical flow and similar algorithms, but I believe they are only for the 2D image space.

Is there something similar for the 3D case, especially when the object of interest fills up most of the FOV of the camera.

edit: this has to be without fiducials

Thank you

Improve this question
$\endgroup$
1
  • $\begingroup$ Are you able to mount something on the object? If so then AR markers could also be an option (this could even work with just one camera). $\endgroup$
    – fibonatic
    Feb 4, 2020 at 15:01

1 Answer 1

Reset to default
1
$\begingroup$

from my experience, you can do it on the 2D image together with the disparity information.

Reason:

  1. tracking based on image has been well developed in recent years, there should be many advanced algorithms you can use directly(may based on NN

  2. you can project the detection result from the 2D image to your disparity to get a distance of the object, which helps:

    2.1. you can get the distance between your robot and target besides the x,y coordinates in camera view, which makes your control easier

    2.2. when the 2D tracking failed, like the result jumped to somewhere else, you can use the distance info to identify this kind of jump

Improve this answer
$\endgroup$
2
  • $\begingroup$ The feature tracking in the image (2D) projected into 3D sounds good. How do you track the 3D points from one frame to another after that? $\endgroup$
    – simplename
    Feb 28, 2020 at 4:40
  • $\begingroup$ @simplename not track actually, should be more like "look up". 1.the disparity is calculated from the stereo image so the timestamp can be aligned with no fault. 2. the project matrix can be calculated since the camera is mounted on rigid body. 3. the disparity inforamtion of the target will be obviously different than the enviorment. So it may be quite easy to just look up a certain place of the dispary map and read its value. The hard part may be when your target is among other stuffs. $\endgroup$
    – lanyusea
    Feb 29, 2020 at 8:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.