Ensembling Uncertainty Measures to Improve Safety of Black-Box Classifiers

TL;DR

This paper introduces SPROUT, a safety wrapper that uses ensembles of uncertainty measures to detect and block misclassifications in black-box classifiers, thereby enhancing system safety across various applications.

Contribution

The paper presents SPROUT, a novel ensemble-based safety wrapper that detects misclassifications in black-box classifiers and prevents their propagation, improving safety in machine learning systems.

Findings

SPROUT detects a large fraction of misclassifications.

It can identify all misclassifications in specific cases.

The implementation is publicly available and easy to deploy.

Abstract

Machine Learning (ML) algorithms that perform classification may predict the wrong class, experiencing misclassifications. It is well-known that misclassifications may have cascading effects on the encompassing system, possibly resulting in critical failures. This paper proposes SPROUT, a Safety wraPper thROugh ensembles of UncertainTy measures, which suspects misclassifications by computing uncertainty measures on the inputs and outputs of a black-box classifier. If a misclassification is detected, SPROUT blocks the propagation of the output of the classifier to the encompassing system. The resulting impact on safety is that SPROUT transforms erratic outputs (misclassifications) into data omission failures, which can be easily managed at the system level. SPROUT has a broad range of applications as it fits binary and multi-class classification, comprising image and tabular datasets. We experimentally show that SPROUT always identifies a huge fraction of the misclassifications of supervised classifiers, and it is able to detect all misclassifications in specific cases. SPROUT implementation contains pre-trained wrappers, it is publicly available and ready to be deployed with minimal effort.