Approaches to macro decompositions of large Markov decision process planning problems

Terran Lane, Leslie Pack Kaelbling, Douglas W. Gage, Howie M. Choset
2002 Mobile Robots XVI  
Mobile robot navigation tasks are subject to motion stochasticity arising from the robot's local controllers, which casts the navigational task into a Markov decision process framework. The MDP may, however, be intractably large; in this work we consider the prioritized package delivery problem which yields an exponentially large state space. We demonstrate that the bulk of this state space is tied to a sub-problem that is an instance of the traveling salesdroid problem and that exponential
more » ... ovements in solution time for the MDP can be achieved by addressing the TSP sub-problem separately. This process produces a suboptimal solution, but we show that the degree of suboptimality can be controlled by employing more effective TSP approximators. The key contribution is the demonstration that MDP solution techniques can substantially benefit from careful application of well-understood deterministic optimization techniques.
doi:10.1117/12.457435 dblp:conf/mr/LaneK01 fatcat:3sset7xnkjb4zoxrxk6gsjlmiy