State-Continuity Approximation of Markov Decision Processes via Finite Element Methods for Autonomous System Planning

TL;DR

This paper introduces a finite element method-based approach for solving continuous-state Markov Decision Processes in autonomous system planning, using moments of transition probabilities to improve decision-making.

Contribution

It presents a novel finite element method to approximate the value function directly from moments, eliminating the need for explicit transition models in continuous MDPs.

Findings

Improved path smoothness over baseline methods

Reduced travel distance and time costs

Validated through extensive simulations

Abstract

Motion planning under uncertainty for an autonomous system can be formulated as a Markov Decision Process with a continuous state space. In this paper, we propose a novel solution to this decision-theoretic planning problem that directly obtains the continuous value function with only the first and second moments of the transition probabilities, alleviating the requirement for an explicit transition model in the literature. We achieve this by expressing the value function as a linear combination of basis functions and approximating the Bellman equation by a partial differential equation, where the value function can be naturally constructed using a finite element method. We have validated our approach via extensive simulations, and the evaluations reveal that to baseline methods, our solution leads to in terms of path smoothness, travel distance, and time costs.