Conformal Decision Theory: Safe Autonomous Decisions from Imperfect Predictions

TL;DR

This paper introduces Conformal Decision Theory, a framework that ensures safe autonomous decisions with statistical guarantees, even under imperfect predictions and adversarial conditions, applicable across robotics, finance, and manufacturing.

Contribution

It extends conformal prediction to directly calibrate decision-making processes, providing provable safety guarantees without assumptions on data distribution or world models.

Findings

Effective in robot motion planning around humans

Applicable to automated stock trading

Enhances safety and reliability in manufacturing processes

Abstract

We introduce Conformal Decision Theory, a framework for producing safe autonomous decisions despite imperfect machine learning predictions. Examples of such decisions are ubiquitous, from robot planning algorithms that rely on pedestrian predictions, to calibrating autonomous manufacturing to exhibit high throughput and low error, to the choice of trusting a nominal policy versus switching to a safe backup policy at run-time. The decisions produced by our algorithms are safe in the sense that they come with provable statistical guarantees of having low risk without any assumptions on the world model whatsoever; the observations need not be I.I.D. and can even be adversarial. The theory extends results from conformal prediction to calibrate decisions directly, without requiring the construction of prediction sets. Experiments demonstrate the utility of our approach in robot motion planning around humans, automated stock trading, and robot manufacturing.