Let's think of the following situations:
- You are teaching a robot to play ping pong
- You are teaching a program to calculate square root
- You are teaching math to a kid in school
These situations (i.e. supervised learning), and many others have one thing (among others) in common: the learner gets a reward based on its performance.
My question is, what should the reward function look like? Is there a "best" answer, or does it depend on the situation? If it depends on the situation, how does one determine which reward function to pick?
For example, take the following three reward functions:
- below a certain point, bad or worse are the same: you get nothing
- there is a clear difference between almost good and perfect
- you get reward linearly proportional to your performance
- if your performance is bad, it's ok, you did your best: you still get some reward
- there is not much difference between perfect and almost good
Intuitively, I'd think
A would make the robot very focused and learn the exact pattern, but become stupid when dealing with similar patterns, while
C would make it more adaptable to change at the cost of losing perfection.
One might also think of more complex functions, just to show but few:
So, how does one know which function to pick? Is it known which behavior would emerge from (at least) the basic
A side question is would this be fundamentally different for robots and human kids?