Overfitting in Histopathology Model Training: The Need for Customized Architectures

TL;DR

This paper highlights the overfitting issues in deep learning models for histopathology and advocates for designing specialized architectures rather than relying on large-scale models from natural image domains.

Contribution

It demonstrates that customized, domain-specific architectures outperform large models in histopathology tasks, especially with limited data.

Findings

Large models often overfit on histopathology data

Customized architectures achieve comparable or better performance

Simpler models reduce overfitting in limited datasets

Abstract

This study investigates the critical problem of overfitting in deep learning models applied to histopathology image analysis. We show that simply adopting and fine-tuning large-scale models designed for natural image analysis often leads to suboptimal performance and significant overfitting when applied to histopathology tasks. Through extensive experiments with various model architectures, including ResNet variants and Vision Transformers (ViT), we show that increasing model capacity does not necessarily improve performance on histopathology datasets. Our findings emphasize the need for customized architectures specifically designed for histopathology image analysis, particularly when working with limited datasets. Using Oesophageal Adenocarcinomas public dataset, we demonstrate that simpler, domain-specific architectures can achieve comparable or better performance while minimizing overfitting.