Maximizing Reach-Avoid Probabilities for Linear Stochastic Systems via Control Architectures

TL;DR

This paper introduces a scalable control architecture combining Model Predictive Control and Markov Decision Processes to maximize reach-avoid probabilities in high-dimensional stochastic systems, demonstrated on a 12D quadcopter.

Contribution

It proposes a novel control framework that integrates MPC with MDPs, enabling scalable and less conservative reach-avoid probability maximization for complex systems.

Findings

Effective on a 12D quadcopter model in cluttered environments

Provides bounds on reach-avoid probability despite approximation errors

Demonstrates scalability and flexibility of the approach

Abstract

The maximization of reach-avoid probabilities for stochastic systems is a central topic in the control literature. Yet, the available methods are either restricted to low-dimensional systems or suffer from conservative approximations. To address these limitations, we propose control architectures that combine the flexibility of Markov Decision Processes with the scalability of Model Predictive Controllers. The Model Predictive Controller tracks reference signals while remaining agnostic to the stochasticity and reach-avoid objective. Instead, the reach-avoid probability is maximized by optimally updating the controller's reference online. To achieve this, the closed-loop system, consisting of the system and Model Predictive Controller, is abstracted as a Markov Decision Process in which a new reference can be chosen at every time-step. A feedback policy generating optimal references is then computed via Dynamic Programming. If the state space of the system is continuous, the Dynamic Programming algorithm must be executed on a finite system approximation. Modifications to the Model Predictive Controller enable a computationally efficient robustification of the Dynamic Programming algorithm to approximation errors, preserving bounds on the achieved reach-avoid probability. The approach is validated on a perturbed 12D quadcopter model in cluttered reach-avoid environments proving its flexibility and scalability.