Kernel Attention Transformer (KAT) for Histopathology Whole Slide Image Classification

TL;DR

This paper introduces Kernel Attention Transformer (KAT), a novel model designed to improve the efficiency and effectiveness of classifying gigapixel histopathology whole slide images by capturing hierarchical context with lower computational costs.

Contribution

The paper proposes a kernel attention mechanism that enhances hierarchical context modeling in WSIs while reducing computational complexity compared to traditional Transformers.

Findings

KAT outperforms 6 state-of-the-art methods on gastric and endometrial datasets.

KAT demonstrates higher accuracy and efficiency in WSI classification.

The method is validated on large-scale datasets with thousands of WSIs.

Abstract

Transformer has been widely used in histopathology whole slide image (WSI) classification for the purpose of tumor grading, prognosis analysis, etc. However, the design of token-wise self-attention and positional embedding strategy in the common Transformer limits the effectiveness and efficiency in the application to gigapixel histopathology images. In this paper, we propose a kernel attention Transformer (KAT) for histopathology WSI classification. The information transmission of the tokens is achieved by cross-attention between the tokens and a set of kernels related to a set of positional anchors on the WSI. Compared to the common Transformer structure, the proposed KAT can better describe the hierarchical context information of the local regions of the WSI and meanwhile maintains a lower computational complexity. The proposed method was evaluated on a gastric dataset with 2040 WSIs and an endometrial dataset with 2560 WSIs, and was compared with 6 state-of-the-art methods. The experimental results have demonstrated the proposed KAT is effective and efficient in the task of histopathology WSI classification and is superior to the state-of-the-art methods. The code is available at https://github.com/zhengyushan/kat.