When normalization hallucinates: unseen risks in AI-powered whole slide image processing

TL;DR

This paper highlights the risks of hallucinations in AI-based whole slide image normalization, demonstrating that current methods can produce artifacts that are hard to detect and may compromise clinical analysis, urging for better validation.

Contribution

The authors introduce a novel image comparison measure to automatically detect hallucinations and systematically evaluate normalization methods on real-world data, exposing their limitations.

Findings

Hallucinations are common in AI-normalized WSIs on real-world data.

Current evaluation metrics often overlook these hallucinations.

Proposed measure effectively detects artifacts not captured by traditional metrics.

Abstract

Whole slide image (WSI) normalization remains a vital preprocessing step in computational pathology. Increasingly driven by deep learning, these models learn to approximate data distributions from training examples. This often results in outputs that gravitate toward the average, potentially masking diagnostically important features. More critically, they can introduce hallucinated content, artifacts that appear realistic but are not present in the original tissue, posing a serious threat to downstream analysis. These hallucinations are nearly impossible to detect visually, and current evaluation practices often overlook them. In this work, we demonstrate that the risk of hallucinations is real and underappreciated. While many methods perform adequately on public datasets, we observe a concerning frequency of hallucinations when these same models are retrained and evaluated on real-world clinical data. To address this, we propose a novel image comparison measure designed to automatically detect hallucinations in normalized outputs. Using this measure, we systematically evaluate several well-cited normalization methods retrained on real-world data, revealing significant inconsistencies and failures that are not captured by conventional metrics. Our findings underscore the need for more robust, interpretable normalization techniques and stricter validation protocols in clinical deployment.