Detecting and Mitigating Test-time Failure Risks via Model-agnostic Uncertainty Learning

TL;DR

This paper presents Risk Advisor, a model-agnostic tool that estimates failure risks and uncertainties of black-box classifiers, helping to identify and mitigate deployment risks by analyzing sources of uncertainty.

Contribution

Introduction of Risk Advisor, a post-hoc meta-learner that decomposes uncertainty into aleatoric and epistemic components for failure risk prediction in black-box models.

Findings

Risk Advisor reliably predicts failure risks across various models and datasets.

It outperforms strong baseline methods in failure risk estimation.

It provides actionable insights by distinguishing sources of uncertainty.

Abstract

Reliably predicting potential failure risks of machine learning (ML) systems when deployed with production data is a crucial aspect of trustworthy AI. This paper introduces Risk Advisor, a novel post-hoc meta-learner for estimating failure risks and predictive uncertainties of any already-trained black-box classification model. In addition to providing a risk score, the Risk Advisor decomposes the uncertainty estimates into aleatoric and epistemic uncertainty components, thus giving informative insights into the sources of uncertainty inducing the failures. Consequently, Risk Advisor can distinguish between failures caused by data variability, data shifts and model limitations and advise on mitigation actions (e.g., collecting more data to counter data shift). Extensive experiments on various families of black-box classification models and on real-world and synthetic datasets covering common ML failure scenarios show that the Risk Advisor reliably predicts deployment-time failure risks in all the scenarios, and outperforms strong baselines.