PATHS: A Hierarchical Transformer for Efficient Whole Slide Image Analysis

TL;DR

PATHS introduces a hierarchical transformer approach that mimics pathologists' examination process, efficiently analyzing whole slide images by selectively focusing on relevant regions, leading to improved prediction performance with less computational cost.

Contribution

The paper presents PATHS, a novel hierarchical transformer model that selectively filters informative patches in WSIs, enabling efficient and interpretable analysis for pathology tasks.

Findings

Outperforms previous methods on TCGA datasets

Processes only a small proportion of the slide

Provides interpretable region importance measures

Abstract

Computational analysis of whole slide images (WSIs) has seen significant research progress in recent years, with applications ranging across important diagnostic and prognostic tasks such as survival or cancer subtype prediction. Many state-of-the-art models process the entire slide - which may be as large as $150,000 \times 150,000$ pixels - as a bag of many patches, the size of which necessitates computationally cheap feature aggregation methods. However, a large proportion of these patches are uninformative, such as those containing only healthy or adipose tissue, adding significant noise and size to the bag. We propose Pathology Transformer with Hierarchical Selection (PATHS), a novel top-down method for hierarchical weakly supervised representation learning on slide-level tasks in computational pathology. PATHS is inspired by the cross-magnification manner in which a human pathologist examines a slide, recursively filtering patches at each magnification level to a small subset relevant to the diagnosis. Our method overcomes the complications of processing the entire slide, enabling quadratic self-attention and providing a simple interpretable measure of region importance. We apply PATHS to five datasets of The Cancer Genome Atlas (TCGA), and achieve superior performance on slide-level prediction tasks when compared to previous methods, despite processing only a small proportion of the slide.