POMDPs extend MDPs by conceiling state and adding an observation model. A POMDP controller processes either
- action/observation histories or
- a bayesian belief state, computed from the observations (belief-MDP transformation)
In a complex, real-world system like a robot, one usually preprocesses sensory readings using filters (Kalmann, HMM, whatever). The result of which is a belief-state.
I am looking for publications that discuss the problem of fitting a (probably more abstract) POMDP model on top of an existing filter-bank.
- Do you have to stick to the belief-MDP, and hand over the filtered belief-state to the controller?
- Is there any way of using history-based POMDP controllers, like MCTS?
- How do you construct/find the abstract observations you need to formulate the POMDP model?