Virchow2: Scaling Self-Supervised Mixed Magnification Models in Pathology

TL;DR

This paper introduces Virchow2, a set of large-scale, domain-specific vision transformer models trained on diverse histopathology data, achieving state-of-the-art results in pathology tasks by scaling data, model size, and applying tailored algorithms.

Contribution

The paper presents new pathology-specific vision transformer models scaled to large data and model sizes, with algorithmic modifications for improved performance in computational pathology.

Findings

Virchow2 models outperform previous models on 12 pathology tasks.

Data diversity and domain-specific methods are crucial for performance.

Scaling data and model size together yields the best results.

Abstract

Foundation models are rapidly being developed for computational pathology applications. However, it remains an open question which factors are most important for downstream performance with data scale and diversity, model size, and training algorithm all playing a role. In this work, we propose algorithmic modifications, tailored for pathology, and we present the result of scaling both data and model size, surpassing previous studies in both dimensions. We introduce three new models: Virchow2, a 632 million parameter vision transformer, Virchow2G, a 1.9 billion parameter vision transformer, and Virchow2G Mini, a 22 million parameter distillation of Virchow2G, each trained with 3.1 million histopathology whole slide images, with diverse tissues, originating institutions, and stains. We achieve state of the art performance on 12 tile-level tasks, as compared to the top performing competing models. Our results suggest that data diversity and domain-specific methods can outperform models that only scale in the number of parameters, but, on average, performance benefits from the combination of domain-specific methods, data scale, and model scale.