Uncertainty estimation for out-of-distribution detection in computational histopathology

TL;DR

This paper evaluates various uncertainty estimation methods in computational histopathology, highlighting a distance-aware approach's superior performance and discussing challenges in out-of-distribution detection for clinical applications.

Contribution

It introduces a distance-aware uncertainty estimation method and assesses its effectiveness compared to existing approaches in histopathology.

Findings

Distance-aware method outperforms Monte Carlo dropout and deep ensembles.

All methods show reduced performance on novel, out-of-distribution samples.

Uncertainty thresholding has limitations for out-of-distribution detection.

Abstract

In computational histopathology algorithms now outperform humans on a range of tasks, but to date none are employed for automated diagnoses in the clinic. Before algorithms can be involved in such high-stakes decisions they need to "know when they don't know", i.e., they need to estimate their predictive uncertainty. This allows them to defer potentially erroneous predictions to a human pathologist, thus increasing their safety. Here, we evaluate the predictive performance and calibration of several uncertainty estimation methods on clinical histopathology data. We show that a distance-aware uncertainty estimation method outperforms commonly used approaches, such as Monte Carlo dropout and deep ensembles. However, we observe a drop in predictive performance and calibration on novel samples across all uncertainty estimation methods tested. We also investigate the use of uncertainty thresholding to reject out-of-distribution samples for selective prediction. We demonstrate the limitations of this approach and suggest areas for future research.