A systematic evaluation of grayscale conversion methods for mitigating color variation in deep learning-based histopathological image analysis

TL;DR

This paper evaluates grayscale conversion methods to reduce the impact of color variation in histopathological image analysis using deep learning.

Contribution

The study introduces a novel attention-based grayscale conversion method (ACSRM) and demonstrates its effectiveness in improving model generalization.

Findings

Grayscale methods achieved performance comparable to RGB in homogeneous settings.

ACSRM outperformed RGB in cross-scanner tests by 0.31 in a specific class.

Luster improved F1-scores from 0.50 to 0.78 in cross-center evaluations.

Abstract

The clinical adoption of deep learning (DL) for histopathological image analysis is hindered by performance degradation caused by color variations arising from disparate staining protocols and scanning technologies. As morphological features may effectively provide the diagnostic information in hematoxylin and eosin slides, this study investigated grayscale conversion as an approach to standardize input for DL. We evaluated six grayscale algorithms against RGB across: (1) a single-center baseline, (2) a mixed multicenter training, (3) a cross-scanner generalization test, and (4) a cross-center generalization test. Furthermore, a novel attention-based grayscale conversion method (ACSRM) was investigated. It utilizes the transformer's attention mechanism to preserve critical color information through long-range pixel dependencies. In homogeneous settings, the best-performing grayscale methods achieved performance comparable to RGB (All-class F1 differences: −0.01 to 0.04 and no intersection over union differences). In mixed-center training, at least one of the grayscale algorithms outperformed RGB in every model, with 23 of 30 model combinations exhibiting statistically distinct decision behaviors (Wilcoxon signed-rank test: p < 0.05). Under the distribution-shift scenario, grayscale methods demonstrated better generalization: ACSRM (with Swin-Transformer-base (Swin-B)) outperformed the RGB baseline by 0.31 (0.14 and 0.45) on a specific class in cross-scanner tests, while demonstrating comparable performance on the remaining classes. Similarly, Luster with Swin-B improved F1-scores from 0.50 to 0.78 in cross-center evaluation. Statistical analysis confirmed significant differences in predictive behavior for these combinations on (3) and (4) (McNemar's Test: p < 0.05). Overall, ACSRM and Luster emerged as the most effective strategies for enhancing DL generalization, facilitating reliable clinical deployment.

Unlabelled Image

•Deep learning models trained on grayscale inputs remain competitive with RGB baselines across homogeneous evaluations.•Grayscale conversion serves as a generalization technique against scanner- and center-induced color variations.•Top-performing grayscale models mostly exhibit statistically distinct decision functions compared to their RGB baselines.•Luster and our novel ACSRM emerge as the most effective strategies for mitigating the color variation challenge.•Limitations include the task-specific evaluations, the focus on RGB-grayscale comparisons, and reliance on patch sampling.

Deep learning models trained on grayscale inputs remain competitive with RGB baselines across homogeneous evaluations.

Grayscale conversion serves as a generalization technique against scanner- and center-induced color variations.

Top-performing grayscale models mostly exhibit statistically distinct decision functions compared to their RGB baselines.

Luster and our novel ACSRM emerge as the most effective strategies for mitigating the color variation challenge.

Limitations include the task-specific evaluations, the focus on RGB-grayscale comparisons, and reliance on patch sampling.

Figures