Reinforcement Learning in global and local path planning for mobile robots and self-driving car

Question

Most RL courses start with grid world problem like this where robot has to navigate from start to end and RL helps in generating the optimum policy. (State-action pairing).

I am not able to relate this to real robotic (or specifically self driving car application). Most self driving car RL application I have read is obtaining the right acceleration and steering angle given the position of the car relative to its environment. (Based on data from vision sensors). This is very different from this grid-world application. So my questions are

1. Is this example, not applicable for self-driving cars or mobile robots?
2. If it is applicable, would it be right to say, this is a global planning application where robot find the path at a broad level for static obstacles? Also, this is not a local planning application where dynamic obstacles and robot actions relative to that cane be determined?

Answer 1

1. No, this is not applicable for a car, it is just an introductory, extremely simplified example. It is one step closer to a mobile robot, then to a car, at least a mobile robot (at least some of them) is capable of moving in any direction (are holomonic), a car could not move instantaneously to the left or right (it is non holomonic).

2. In general as a first step, a global plan is planned. But this is nothing new, your navigation system does the same, finds a global path to your destination. Reinforcement learning can be used to make sure that the robot/car stays on that path and avoids obstacles. Control theory can also be applied to make sure the robot stays on the path, but it just maintains the trajectory, does not avoid obstacles (well in many cases does not avoid obstacles, olne could argue that advanced control theory methods, like model based predictive control can avoid obstacles).

Answer 2

1- No it is not. This example is the beginning of RL, while Self-driving cars are way much complex. In a simplified view, there are two main differences. state and action in the shown example are discrete, while for a real robotic application are continuous.

2- Despite the promising results, still RL is far from global path planning -planning to long ahead- in partially observable environments -like for a self-driving car -. The best way to utilize RL is to use a classic path planning method, to find the waypoints toward the goal and feed those waypoints to the RL module as a trajectory following.