Distribution-Free Uncertainty Quantification for Continuous AI Agent Evaluation

TL;DR

This paper introduces distribution-free methods for uncertainty quantification in continuous AI agent evaluation, ensuring reliable coverage guarantees and robust multi-agent pipeline assessments.

Contribution

It adapts conformal prediction techniques to continuous evaluation, develops compositional uncertainty bounds, and introduces abstention rules with FDR correction, validated on real-world data.

Findings

Conformal intervals achieve calibration error below 0.02 at 24h horizon.

ACI adjusts intervals by 35% after agent updates and then reconverges.

Per-agent coverage is concentrated around the nominal level, with divergence predicting instability.

Abstract

We adapt split conformal prediction and adaptive conformal inference (ACI) to continuous AI agent evaluation, providing distribution-free coverage guarantees for forecasted quality scores. Conformal intervals achieve calibration error below 0.02 across all nominal levels at the 24h horizon, while ACI correctly widens intervals by 35% following agent releases then reconverges. We further develop compositional uncertainty bounds for multi-agent pipelines (validated via simulation across inter-stage correlations rho in [-0.5, 0.9]), a conformal abstention rule for pairwise rankings with controlled false-ranking rate, and FDR-corrected abstention for leaderboard-scale multiple testing. Evaluating 50 agents via 18 real-time signals collected hourly, we show that per-agent conditional coverage is well-concentrated around the nominal level (mean 80.4%, 90% of agents within [72%, 90%]), and that cross-source sentiment divergence predicts ranking instability (r=0.64, p<0.01). A circularity-controlled validation confirms the framework captures signal beyond benchmarks (rho_s=0.52, p<0.01, n=35). Code and data are released under CC BY 4.0.