Rethinking Machine Learning Model Evaluation in Pathology

TL;DR

This paper proposes practical guidelines for evaluating machine learning models in pathology to ensure robustness, interpretability, and clinical relevance, addressing current challenges like variability and domain shift.

Contribution

It introduces a comprehensive evaluation framework tailored for pathology ML models, including measures for variability, domain shift, and confounding factors.

Findings

Guidelines improve robustness assessment in pathology ML models

Framework facilitates better domain adaptation and confounder detection

Enhances clinical trust in machine learning applications

Abstract

Machine Learning has been applied to pathology images in research and clinical practice with promising outcomes. However, standard ML models often lack the rigorous evaluation required for clinical decisions. Machine learning techniques for natural images are ill-equipped to deal with pathology images that are significantly large and noisy, require expensive labeling, are hard to interpret, and are susceptible to spurious correlations. We propose a set of practical guidelines for ML evaluation in pathology that address the above concerns. The paper includes measures for setting up the evaluation framework, effectively dealing with variability in labels, and a recommended suite of tests to address issues related to domain shift, robustness, and confounding variables. We hope that the proposed framework will bridge the gap between ML researchers and domain experts, leading to wider adoption of ML techniques in pathology and improving patient outcomes.