Online Planning of Uncertain MDPs under Temporal Tasks and Safe-Return Constraints

TL;DR

This paper presents an online planning framework for mobile robots operating under uncertain MDP models and complex temporal tasks, ensuring safety and high-probability return to home states through Bayesian exploration and policy optimization.

Contribution

It introduces a novel approach combining Bayesian model updating, temporal logic task satisfaction, and safe-return constraints for uncertain MDPs.

Findings

The framework guarantees safety and task satisfaction with high probability.

Numerical simulations validate the effectiveness of the proposed methods.

The approach outperforms traditional greedy exploration under ergodicity assumptions.

Abstract

This paper addresses the online motion planning problem of mobile robots under complex high-level tasks. The robot motion is modeled as an uncertain Markov Decision Process (MDP) due to limited initial knowledge, while the task is specified as Linear Temporal Logic (LTL) formulas. The proposed framework enables the robot to explore and update the system model in a Bayesian way, while simultaneously optimizing the asymptotic costs of satisfying the complex temporal task. Theoretical guarantees are provided for the synthesized outgoing policy and safety policy. More importantly, instead of greedy exploration under the classic ergodicity assumption, a safe-return requirement is enforced such that the robot can always return to home states with a high probability. The overall methods are validated by numerical simulations.