4
$\begingroup$

Vision is important part of robotics and frequently it is unavoidable component of control loop. E.g. many clothes/garment handling algorithms rely on visual cues in deciding how to proceed. The question is - does simulation environments (Gazebo or some others) allow one to design world with robot and garment and simulate not only garment dynamics but simulate also what robot sees, how robot perceives garment in each simulation step? If it is not possible to simulate vision then how to simulate algorithms with vision as component of control loop?

Maybe simulation of vision can be good research theme? Are here some trend or good articles about it? Some initial projects that could be expanded?

Actually - it can be stated as more general question - is it possible to simulate sensors in Gazebo? E.g. food handling (soft-body handling) can involve tactile sensors. In principle Gazebo can calculate deformation and forces of soft-body and format these data as the simulated values of sensor readings. Maybe similar mechanism can be used for simulation of vision as well?

Improve this question
$\endgroup$
2
  • $\begingroup$ yes. I have tried usb cameras and Kinect $\endgroup$
    – lanyusea
    Jul 23, 2015 at 16:46
  • 1
    $\begingroup$ Alternatively V-REP simulator does. $\endgroup$
    – acs
    Jul 24, 2015 at 21:00

1 Answer 1

Reset to default
1
$\begingroup$

While Gazebo - and most other simulators - will allow you to simulate sensors, you have to realize that this the "weak" part of realtime simulators. Making an accurate simulated camera is really difficult. The result would therefore be great to test your integration and basic control algorithm behaviour (for instance that you move right when you should), but very limited to test the vision-based algorithm itself, as well as test fine print of the control laws.

The strategy I follow in these cases is to (1) create a "simulated component simulation" to test the control system in the simulator and (2) use log replay to test the vision component:

  1. Create a "vision component simulator". This component would output what the vision part of your system should output, but within the simulated environment and WITHOUT processing any image. Add noise / outliers as necessary in the simulated output.

  2. Use a framework that allows you to log data and replay it offline (as Rock or ROS do). You would then have test cases for your vision component that would test its behaviour in way that the simulator could never dream to achieve.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.