Monitoring Model Deterioration with Explainable Uncertainty Estimation via Non-parametric Bootstrap

TL;DR

This paper introduces a novel non-parametric bootstrap approach combined with SHAP values to monitor and explain model deterioration in deployment environments without relying on labeled data, outperforming existing methods.

Contribution

It proposes a new bootstrap-based uncertainty estimation method and an explainable framework for detecting and understanding model deterioration without labeled data.

Findings

Our method outperforms state-of-the-art in deterioration detection.

The approach provides explainable insights into model failure causes.

Open source package 'doubt' implements the proposed techniques.

Abstract

Monitoring machine learning models once they are deployed is challenging. It is even more challenging to decide when to retrain models in real-case scenarios when labeled data is beyond reach, and monitoring performance metrics becomes unfeasible. In this work, we use non-parametric bootstrapped uncertainty estimates and SHAP values to provide explainable uncertainty estimation as a technique that aims to monitor the deterioration of machine learning models in deployment environments, as well as determine the source of model deterioration when target labels are not available. Classical methods are purely aimed at detecting distribution shift, which can lead to false positives in the sense that the model has not deteriorated despite a shift in the data distribution. To estimate model uncertainty we construct prediction intervals using a novel bootstrap method, which improves upon the work of Kumar & Srivastava (2012). We show that both our model deterioration detection system as well as our uncertainty estimation method achieve better performance than the current state-of-the-art. Finally, we use explainable AI techniques to gain an understanding of the drivers of model deterioration. We release an open source Python package, doubt, which implements our proposed methods, as well as the code used to reproduce our experiments.