Drift-to-Action Controllers: Budgeted Interventions with Online Risk Certificates

TL;DR

This paper introduces Drift2Act, a controller that uses online risk certificates to make safe, budgeted interventions in machine learning systems experiencing distribution drift, improving safety and recovery.

Contribution

It presents Drift2Act, a novel approach combining drift sensing and active risk certification for reliable, cost-effective drift response in streaming ML systems.

Findings

Achieves near-zero safety violations in experiments.

Outperforms alarm-only and schedule-based methods.

Provides fast recovery with moderate costs.

Abstract

Deployed machine learning systems face distribution drift, yet most monitoring pipelines stop at alarms and leave the response underspecified under labeling, compute, and latency constraints. We introduce Drift2Act, a drift-to-action controller that treats monitoring as constrained decision-making with explicit safety. Drift2Act combines a sensing layer that maps unlabeled monitoring signals to a belief over drift types with an active risk certificate that queries a small set of delayed labels from a recent window to produce an anytime-valid upper bound $U_t(\delta)$ on current risk. The certificate gates operation: if $U_t(\delta) \le \tau$, the controller selects low-cost actions (e.g., recalibration or test-time adaptation); if $U_t(\delta) > \tau$, it activates abstain/handoff and escalates to rollback or retraining under cooldowns. In a realistic streaming protocol with label delay and explicit intervention costs, Drift2Act achieves near-zero safety violations and fast recovery at moderate cost on WILDS Camelyon17, DomainNet, and a controlled synthetic drift stream, outperforming alarm-only monitoring, adapt-always adaptation, schedule-based retraining, selective prediction alone, and an ablation without certification. Overall, online risk certification enables reliable drift response and reframes monitoring as decision-making with safety.