1
$\begingroup$

I'm looking to learn more about this sim-to-real gap. I'm new to the concept and am trying to understand what to look for. Most notably i'm curious about how well an object detection algorithm trained based on simulation data would transfer over to reality and vice versa. Should I mix the training and test datasets with both sim and real data? How much does the performance differ? Will I be crippling the accuracy of my real object recognition by polluting the training dataset with some simulation data? Looking for some guidance.

$\endgroup$
  • 1
    $\begingroup$ this question may be better suited for ai.stackexchange.com $\endgroup$ – 50k4 Oct 12 at 21:07
  • $\begingroup$ Welcome to Robotics Nathan Boyd, but I'm afraid that this question is too broad. We prefer practical, answerable questions based on actual problems that you face, so it's a good idea to include details of what you want to understand, what you've researched so far, what you found & what you expected to find. Please take a look at How to Ask & tour for more information on how stack exchange works and work through the Robotics question checklist to edit your question to make it clearer. $\endgroup$ – Ben Oct 14 at 13:31
0
$\begingroup$

Even though it is very interesting, your question is quite broad. I will not cover all the details, but give you an overview.

Also, there are some open questions to your question.

  1. What do you simulate? The entire robot? Only the AI? Only the hardware, with real AI? The target? The rest of the environment? A mix of these? Which mix?

  2. Simulation and reality can be as related or as independent as you want. In "serious" projects, the simulation improves the "reality", and the reality is used to improve the simulation environment. How these are done, depends on the specifics of the project and the genius-ness of the people working on it.

Will I be crippling the accuracy of my real object recognition by polluting the training dataset with some simulation data?

As I mentioned earlier, that is where the genius-ness (or strike of luck) come into play. It depends how you mix and separate the data - you might pollute and destroy, but you might also improve and "educate-to-maturity".

| improve this answer | |
$\endgroup$
0
$\begingroup$

i'm curious about how well an object detection algorithm trained based on simulation data would transfer over to reality and vice versa

It depends on how accurately your simulator reflects reality on the metrics that matter.

Will I be crippling the accuracy of my real object recognition by polluting the training dataset with some simulation data?

If your simulator doesn't generate data that accurately reflects reality.

| improve this answer | |
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.