Active Calibration of Reachable Sets Using Approximate Pick-to-Learn

TL;DR

This paper introduces Approximate Pick-to-Learn, an active learning approach for calibrating reachable sets efficiently, reducing sample requirements and improving accuracy with probabilistic guarantees, demonstrated on a drone racing simulation.

Contribution

It adapts the Pick-to-Learn algorithm for active learning in reachability calibration, integrating conformal prediction for probabilistic guarantees, and demonstrates improved efficiency and accuracy.

Findings

Requires fewer samples than baselines for calibration.

Provides tighter probabilistic guarantees via conformal prediction.

Achieves more accurate reachable sets in drone racing simulation.

Abstract

Reachability computations that rely on learned or estimated models require calibration in order to uphold confidence about their guarantees. Calibration generally involves sampling scenarios inside the reachable set. However, producing reasonable probabilistic guarantees may require many samples, which can be costly. To remedy this, we propose that calibration of reachable sets be performed using active learning strategies. In order to produce a probabilistic guarantee on the active learning, we adapt the Pick-to-Learn algorithm, which produces generalization bounds for standard supervised learning, to the active learning setting. Our method, Approximate Pick-to-Learn, treats the process of choosing data samples as maximizing an approximate error function. We can then use conformal prediction to ensure that the approximate error is close to the true model error. We demonstrate our technique for a simulated drone racing example in which learning is used to provide an initial guess of the reachable tube. Our method requires fewer samples to calibrate the model and provides more accurate sets than the baselines. We simultaneously provide tight generalization bounds.