Conformal Prediction Intervals for Markov Decision Process Trajectories

TL;DR

This paper develops conformal prediction intervals for future trajectories of autonomous systems in Markov Decision Processes, providing probabilistic guarantees on their behavior for safety and reliability.

Contribution

It extends conformal prediction methods to MDP trajectories, combining quantile regression with conformal corrections for reliable future behavior intervals.

Findings

Guarantees that observed trajectories lie within intervals with probability 1-δ

Applied method to invasive species management MDPs

Demonstrated approach on StarCraft2 battles

Abstract

Before delegating a task to an autonomous system, a human operator may want a guarantee about the behavior of the system. This paper extends previous work on conformal prediction for functional data and conformalized quantile regression to provide conformal prediction intervals over the future behavior of an autonomous system executing a fixed control policy on a Markov Decision Process (MDP). The prediction intervals are constructed by applying conformal corrections to prediction intervals computed by quantile regression. The resulting intervals guarantee that with probability $1-\delta$ the observed trajectory will lie inside the prediction interval, where the probability is computed with respect to the starting state distribution and the stochasticity of the MDP. The method is illustrated on MDPs for invasive species management and StarCraft2 battles.