0
$\begingroup$

I have a question. I have found some papers, where they use a Mono-VIO to recover depth of a scene. My question is: Suppose I have a robot arm and attach a mono-VIO system to it. Then I move this arm once over my desk, which is about 1m long. Is it possible to get depth information with metric scaling about the objects lying on my desk from the received data of the Mono-VIO system? With a stereo camera, a snapshot of my desk may be sufficient to accomplish this.

Can someone tell me if this works or if the drift is too high or the accuracy is too low.

$\endgroup$
0
$\begingroup$

You are basically describing an object 3D-reconstruction device. Though usually they use a turntable and fixed camera, but the principle is the exact same. See how this dataset were created. I also know for sure that people sometimes instead use a robotic arm rather than a turntable, so your idea is definitely feasible. The field you want to look into is called multi-view stereo.

I am just going to assume you can find a working multi-view stereo algorithm. So you need to find a way to estimate the positions between the cameras. A Mono-VIO algorithm would work, but is probably not the best solution. One problem is that many Mono-VIO algorithms assume they are always moving to a new area. Basically it forgets past data, because it is irrelevant to the current position. However, in your case the camera is moving in a limited area(the span of the table). So you would you actually want your algorithm to store all the data related to the table, and just use that for pose estimation. They also tend to drift over time, which is again related to not using past data. If you are familiar with SLAM/VIO terms, you want your algorithm to always be doing localization, rather than odometry.

If you do decide to still go the Mono-VIO route then I would recommend ORB-SLAM3. It actually does do localization, so it should hopefully mitigate the drifting problem. However, it would still probably do a large amount of irrelevant work.

As I said before this is actually a pretty common sort of device to build. So you do have other potential options.

  1. Use the robot arm positioning system. If you have a high quality robot arm with good encoders, then it will know how far it moves better than any Mono-VIO system can estimate. You need to calibrate your camera/robot arm together using Hand-eye calibration software, and now when your robot moves you also know how far the camera moves.

  2. Use fiduciary markers like aruco/April tags. Basically put a couple of markers around the table. Your camera can then estimate its position relative to those markers.

  3. Build custom VIO/SLAM system that does more relocalization, and is more customized to your specific problem.

To answer your questions:

Is it possible to get depth information with metric scaling about the objects lying on my desk from the received data of the Mono-VIO system?

Yes.

Can someone tell me if this works or if the drift is too high or the accuracy is too low.

Could work with the right Mono-VIO algorithm. A specialized version ,however, would be better.

Overall I would recommend options 1 or 2. Option 3 or an off the shelf Mono-VIO algorithm should only be attempted if you have to.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.