Hierarchical Decision-Making under Uncertainty: A Hybrid MDP and Chance-Constrained MPC Approach

TL;DR

This paper introduces a hierarchical decision-making framework combining Hybrid Markov Decision Processes and chance-constrained Model Predictive Control to improve safety and efficiency in autonomous driving under uncertainty.

Contribution

It presents a novel integrated approach that models multi-modal environmental uncertainties and ensures safety through joint chance constraints within a unified optimization framework.

Findings

Framework guarantees recursive feasibility and stability.

Enhanced safety and efficiency demonstrated in highway and urban tests.

Outperforms rule-based baseline in various scenarios.

Abstract

This paper presents a hierarchical decision-making framework for autonomous systems operating under uncertainty, demonstrated through autonomous driving as a representative application. Surrounding agents are modeled using Hybrid Markov Decision Processes (HMDPs) that jointly capture maneuver-level and dynamic-level uncertainties, enabling the multi-modal environmental prediction. The ego agent is modeled using a separate HMDP and integrated into a Model Predictive Control (MPC) framework that unifies maneuver selection with dynamic feasibility within a single optimization. A set of joint chance constraints serves as the bridge between environmental prediction and optimization, incorporating multi-modal environment predictions into the MPC formulation and ensuring safety across all plausible interaction scenarios. The proposed framework provides theoretical guarantees on recursive feasibility and asymptotic stability, and its benefits in terms of safety and efficiency are validated through comprehensive evaluations in highway and urban environments, together with comparisons against a rule-based baseline.