Deformable Attention Graph Representation Learning for Histopathology Whole Slide Image Analysis

TL;DR

This paper introduces a deformable attention graph neural network framework for histopathology image analysis, effectively capturing spatial dependencies and morphological relevance in whole slide images, leading to improved classification performance.

Contribution

It presents a novel GNN with deformable attention that dynamically models spatial relationships using learnable offsets, surpassing static graph methods in pathology image analysis.

Findings

Achieved state-of-the-art results on four benchmark datasets.

Enhanced spatial modeling improves classification accuracy.

Demonstrated the effectiveness of deformable attention in capturing complex tissue structures.

Abstract

Accurate classification of Whole Slide Images (WSIs) and Regions of Interest (ROIs) is a fundamental challenge in computational pathology. While mainstream approaches often adopt Multiple Instance Learning (MIL), they struggle to capture the spatial dependencies among tissue structures. Graph Neural Networks (GNNs) have emerged as a solution to model inter-instance relationships, yet most rely on static graph topologies and overlook the physical spatial positions of tissue patches. Moreover, conventional attention mechanisms lack specificity, limiting their ability to focus on structurally relevant regions. In this work, we propose a novel GNN framework with deformable attention for pathology image analysis. We construct a dynamic weighted directed graph based on patch features, where each node aggregates contextual information from its neighbors via attention-weighted edges. Specifically, we incorporate learnable spatial offsets informed by the real coordinates of each patch, enabling the model to adaptively attend to morphologically relevant regions across the slide. This design significantly enhances the contextual field while preserving spatial specificity. Our framework achieves state-of-the-art performance on four benchmark datasets (TCGA-COAD, BRACS, gastric intestinal metaplasia grading, and intestinal ROI classification), demonstrating the power of deformable attention in capturing complex spatial structures in WSIs and ROIs.