Applications of Reinforcement Learning

Question

Many global-control applications of robotics involve incomplete world information and at best can be represented as a POMDP.

Given this, can we really apply RL to most robotics applications? From what I have seen RL almost always needs full state information to work even remotely well.

What areas of robotics do you have sufficient state information to use RL in? I have my doubts on its tractability for global planning so what subproblems work for RL and how can they be integrated into a global planning framework?

Answer 1

It's true that using RL in robotics involves many challenges, including the usually high dimensionality of problem spaces, the cost and limitations of real-world sessions, the impossibility or perfectly modelling the robot-environment system, and the complexity of reward functions that accurately reflect desired behaviors.

That said, a number of approaches have been proposed to address these issues: reducing space dimensionality through discretization or abstraction, enhance learning models with domain information, provide prior knowledge through demonstration and clever use of simulations, and handle uncertainties through probabilistic techniques.

There have been in fact many successful RL experiments involving robot navigation (both ground and aerial) as well as manipulator control. I recommend you check the following articles for an overview and further references:

• Kober, Jens, J. Andrew Bagnell, and Jan Peters. "Reinforcement learning in robotics: A survey." The International Journal of Robotics Research 32.11 (2013): 1238-1274.
• Kormushev, Petar, Sylvain Calinon, and Darwin G. Caldwell. "Reinforcement learning in robotics: Applications and real-world challenges." Robotics 2.3 (2013): 122-148.