Dual Cross-Attention Siamese Transformer for Rectal Tumor Regrowth Assessment in Watch-and-Wait Endoscopy

TL;DR

This paper introduces a Siamese Swin Transformer with Dual Cross-Attention (SSDCA) for accurately detecting rectal tumor regrowth during watch-and-wait endoscopy follow-up, leveraging pretrained transformers for robust feature extraction.

Contribution

The study presents a novel SSDCA model that combines longitudinal endoscopic images without spatial alignment, achieving high accuracy and robustness in detecting tumor regrowth.

Findings

SSDCA achieved 81.76% balanced accuracy, 90.07% sensitivity, and 72.86% specificity.

Model demonstrated stable performance across artifacts like blood, stool, and poor image quality.

UMAP analysis confirmed discriminative feature representations with maximal inter-cluster separation.

Abstract

Increasing evidence supports watch-and-wait (WW) surveillance for patients with rectal cancer who show clinical complete response (cCR) at restaging following total neoadjuvant treatment (TNT). However, objectively accurate methods to early detect local regrowth (LR) from follow-up endoscopy images during WW are essential to manage care and prevent distant metastases. Hence, we developed a Siamese Swin Transformer with Dual Cross-Attention (SSDCA) to combine longitudinal endoscopic images at restaging and follow-up and distinguish cCR from LR. SSDCA leverages pretrained Swin transformers to extract domain agnostic features and enhance robustness to imaging variations. Dual cross attention is implemented to emphasize features from the two scans without requiring any spatial alignment of images to predict response. SSDCA as well as Swin-based baselines were trained using image pairs from 135 patients and evaluated on a held-out set of image pairs from 62 patients. SSDCA produced the best balanced accuracy (81.76\% $\pm$ 0.04), sensitivity (90.07\% $\pm$ 0.08), and specificity (72.86\% $\pm$ 0.05). Robustness analysis showed stable performance irrespective of artifacts including blood, stool, telangiectasia, and poor image quality. UMAP clustering of extracted features showed maximal inter-cluster separation (1.45 $\pm$ 0.18) and minimal intra-cluster dispersion (1.07 $\pm$ 0.19) with SSDCA, confirming discriminative representation learning.