On Continuous Monitoring of Risk Violations under Unknown Shift

TL;DR

This paper introduces a real-time monitoring framework for detecting risk violations in machine learning systems under unpredictable data shifts, ensuring safety guarantees without strict assumptions.

Contribution

It presents a novel sequential hypothesis testing method based on 'testing by betting' for continuous risk monitoring in dynamic environments.

Findings

Effective detection of risk violations under various data shifts

Controlled false alarm rate in risk monitoring

Broad applicability to different risk types and models

Abstract

Machine learning systems deployed in the real world must operate under dynamic and often unpredictable distribution shifts. This challenges the validity of statistical safety assurances on the system's risk established beforehand. Common risk control frameworks rely on fixed assumptions and lack mechanisms to continuously monitor deployment reliability. In this work, we propose a general framework for the real-time monitoring of risk violations in evolving data streams. Leveraging the 'testing by betting' paradigm, we propose a sequential hypothesis testing procedure to detect violations of bounded risks associated with the model's decision-making mechanism, while ensuring control on the false alarm rate. Our method operates under minimal assumptions on the nature of encountered shifts, rendering it broadly applicable. We illustrate the effectiveness of our approach by monitoring risks in outlier detection and set prediction under a variety of shifts.