Semantics-Aware Attention Guidance for Diagnosing Whole Slide Images

TL;DR

This paper introduces Semantics-Aware Attention Guidance (SAG), a novel framework that enhances whole slide image diagnosis by integrating semantic information into attention mechanisms, significantly improving accuracy and interpretability in cancer detection.

Contribution

The paper proposes a new SAG framework that converts diagnostic entities into attention signals and employs a flexible attention loss, advancing the accuracy and interpretability of attention-based models in digital pathology.

Findings

Consistent improvements in accuracy, precision, and recall across two cancer datasets.

Qualitative analysis shows models focus on diagnostically relevant regions.

SAG enhances existing attention-based diagnostic models' performance.

Abstract

Accurate cancer diagnosis remains a critical challenge in digital pathology, largely due to the gigapixel size and complex spatial relationships present in whole slide images. Traditional multiple instance learning (MIL) methods often struggle with these intricacies, especially in preserving the necessary context for accurate diagnosis. In response, we introduce a novel framework named Semantics-Aware Attention Guidance (SAG), which includes 1) a technique for converting diagnostically relevant entities into attention signals, and 2) a flexible attention loss that efficiently integrates various semantically significant information, such as tissue anatomy and cancerous regions. Our experiments on two distinct cancer datasets demonstrate consistent improvements in accuracy, precision, and recall with two state-of-the-art baseline models. Qualitative analysis further reveals that the incorporation of heuristic guidance enables the model to focus on regions critical for diagnosis. SAG is not only effective for the models discussed here, but its adaptability extends to any attention-based diagnostic model. This opens up exciting possibilities for further improving the accuracy and efficiency of cancer diagnostics.