Enhanced Diagnostic Performance via Large-Resolution Inference Optimization for Pathology Foundation Models

TL;DR

This paper introduces an efficient inference method for pathology models that enables high-resolution whole-slide image analysis without excessive GPU memory use, improving accuracy and speed.

Contribution

It proposes a novel space- and time-efficient inference strategy that sparsifies attention and filters tokens, allowing high-resolution WSI processing within existing GPU constraints.

Findings

Up to 7.67% improvement in ROI classification accuracy

Reduces GPU memory and runtime during high-resolution inference

Maintains or improves downstream segmentation performance

Abstract

Despite their prominent performance on tasks such as ROI classification and segmentation, many pathology foundation models remain constrained by a specific input size e.g. 224 x 224, creating substantial inefficiencies when applied to whole-slide images (WSIs), which span thousands of resolutions. A naive strategy is to either enlarge inputs or downsample the WSIs. However, enlarging inputs results in prohibitive GPU memory consumption, while downsampling alters the microns-per-pixel resolution and obscures critical morphological details. To overcome these limitations, we propose an space- and time- efficient inference strategy that sparsifies attention using spatially aware neighboring blocks and filters out non-informative tokens through global attention scores. This design substantially reduces GPU memory and runtime during high-resolution WSI inference while preserving and even improving the downstream performance, enabling inference at higher resolutions under the same GPU budget. The experimental results show that our method can achieves up to an 7.67% improvement in the ROI classification and compatible results in segmentation.