POMDPs are used when we cannot observe all the states. However, I cannot figure out when these POMDPs can be useful in robotics. What is a good example of the use of POMDPs? (I have read one paper where they used them, but I didn't find it obvious why pomdps should be used) What would be good projects ideas based on POMDPs?
1
-
$\begingroup$ POMDP = partially observable Markov decision process? $\endgroup$– AndrewCommented Dec 8, 2013 at 21:26
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
1
A good rule of thumb is that where ever an MDP is useful in theory a POMDP would likely need to be used in reality.
To answer your question directly I would direct you to some of the latest work coming out of the Algorithmic Robotics Lab. My advisor and I recently developed a method wherein we use a POMDP at the core of a new grey-box system identification method. In this method we know an approximate kinematic or dynamic model of the robot in question. We do not however know some of the associated parameters (e.g. mass, center of mass). One way to learn the parameters is to use an extended Kalman filter (EKF) to monitor the system while the motor babbling. However more can be learned if the dynamics are excited correctly. To achieve this part we use a POMDP to plan controls. The details can be found in our paper Online Parameter Estimation via Real-time Replanning of Continuous Gaussian POMDPs.
answered Dec 3, 2013 at 18:10
-
1$\begingroup$ The first sentence nailed it completely. Robots in the real world are never quite able to get all the data they need about their environment, and you need to account for that. $\endgroup$– IanCommented Dec 5, 2013 at 16:10
$\begingroup$
$\endgroup$
Partially Observable MDPs actually are very interesting to robotics, and especially for the partially observable part. Sensor signals (laser, sonar, vision, ..) are noisy, some more then others depending on several variables. We cannot trust the sensor signal completely, but we can use them to estimate the real state. The POMDP can thus be used in robotics to do planning with noisy observations. A list of applications is shown by Cassandra [1], a more general overview of planning with uncertainty in robotics is given by Thrun et al. [2], and of course there are lots of examples on the web. The big problem of POMDPs is that finding their optimal solution is intractable (i.e it easily takes too long to compute), but there are several methods which approximate the optimal solution.
[1] Cassandra, A. R. (1997). A Survey of POMDP Applications. In Uncertainty in Artificial Intelligence (pp. 472–480). Retrieved from http://www.cassandra.org/pomdp/papers/applications.pdf.
[2] Thrun, S., Burgard, W., & Fox, D. (2005). Probabilistic Robotics (Intelligent Robotics and Autonomous Agents. The MIT Press.
