Merging-Diverging Hybrid Transformer Networks for Survival Prediction in Head and Neck Cancer

TL;DR

This paper introduces XSurv, a hybrid transformer network that fuses multi-modality PET-CT images and extracts region-specific features for improved survival prediction in head and neck cancer patients.

Contribution

It proposes a novel merging-diverging transformer framework with specialized attention blocks for multi-modality fusion and region-specific feature extraction, advancing survival prediction methods.

Findings

XSurv outperforms existing methods on HECKTOR 2022 dataset.

The merging encoder effectively fuses PET and CT features.

The diverging decoder extracts prognostic information from tumor regions.

Abstract

Survival prediction is crucial for cancer patients as it provides early prognostic information for treatment planning. Recently, deep survival models based on deep learning and medical images have shown promising performance for survival prediction. However, existing deep survival models are not well developed in utilizing multi-modality images (e.g., PET-CT) and in extracting region-specific information (e.g., the prognostic information in Primary Tumor (PT) and Metastatic Lymph Node (MLN) regions). In view of this, we propose a merging-diverging learning framework for survival prediction from multi-modality images. This framework has a merging encoder to fuse multi-modality information and a diverging decoder to extract region-specific information. In the merging encoder, we propose a Hybrid Parallel Cross-Attention (HPCA) block to effectively fuse multi-modality features via parallel convolutional layers and cross-attention transformers. In the diverging decoder, we propose a Region-specific Attention Gate (RAG) block to screen out the features related to lesion regions. Our framework is demonstrated on survival prediction from PET-CT images in Head and Neck (H&N) cancer, by designing an X-shape merging-diverging hybrid transformer network (named XSurv). Our XSurv combines the complementary information in PET and CT images and extracts the region-specific prognostic information in PT and MLN regions. Extensive experiments on the public dataset of HEad and neCK TumOR segmentation and outcome prediction challenge (HECKTOR 2022) demonstrate that our XSurv outperforms state-of-the-art survival prediction methods.