Let's think of the following situations:
- You are teaching a robot to play ping pong
- You are teaching a program to calculate square root
- You are teaching math to a kid in school
These situations (i.e. supervised learning), and many others have one thing (among others) in common: the learner gets a reward based on its performance.
My question is, what should the reward function look like? Is there a "best" answer, or does it depend on the situation? If it depends on the situation, how does one determine which reward function to pick?
For example, take the following three reward functions:
- Function
A
says:- below a certain point, bad or worse are the same: you get nothing
- there is a clear difference between almost good and perfect
- Function
B
says:- you get reward linearly proportional to your performance
- Function
C
says:- if your performance is bad, it's ok, you did your best: you still get some reward
- there is not much difference between perfect and almost good
Intuitively, I'd think A
would make the robot very focused and learn the exact pattern, but become stupid when dealing with similar patterns, while C
would make it more adaptable to change at the cost of losing perfection.
One might also think of more complex functions, just to show but few:
So, how does one know which function to pick? Is it known which behavior would emerge from (at least) the basic A
, B
and C
functions?
A side question is would this be fundamentally different for robots and human kids?
A
, the robot could become extremely good at the exact task, but terrible at tasks that are similar but slightly different. That's just my guess though. $\endgroup$ – Shahbaz May 7 '13 at 14:14X
gave me the best result", even if not perfectly correct, would give a great rule of thumb. $\endgroup$ – Shahbaz May 7 '13 at 16:13