# Predicting Recurrence in Locally Advanced Rectal Cancer Using Multitask Deep Learning and Multimodal MRI

## TL;DR

This study introduces MultiRecNet, a deep learning model that automatically predicts cancer recurrence in rectal cancer patients using MRI data and clinical information.

## Contribution

The novelty lies in a multitask deep learning framework that integrates multimodal MRI and clinical data for end-to-end survival prediction in rectal cancer.

## Key findings

- The best model achieved a 0.92 C-index for disease-free survival prediction in the internal testing set.
- The model maintained strong performance in the external testing set with a 0.81 C-index for survival prediction.
- MultiRecNet successfully combined segmentation, classification, and survival prediction tasks in a single framework.

## Abstract

To develop and validate a deep multitask network, MultiRecNet, for fully automatic prediction of disease-free survival (DFS) in patients with neoadjuvant chemoradiotherapy (nCRT)–treated locally advanced rectal cancer (LARC).

This retrospective study collected clinical information and baseline multimodal MRI (T2, apparent diffusion coefficient [ADC], Dapp, and Kapp) data from patients with LARC after nCRT at three centers between October 2011 and May 2019. Patients from centers 1 and 2 were divided into training, validation, and internal testing sets, while patients from center 3 served as the external testing set. MultiRecNet is capable of simultaneously performing segmentation, classification, and survival prediction tasks within a single framework. Multiple combinations of data from different clinical stages (pretreatment and postoperative) were input into MultiRecNet to generate different models and identify the model with optimal performance. Evaluation metrics included the Dice similarity coefficient (DSC), the area under the receiver operating characteristic curve (AUC), and the Harrell concordance index (C-index) for the segmentation, classification, and survival prediction tasks, respectively.

The study included 445 patients: 261 in the training set (median age, 60 years [IQR, 53–67 years]; 172 male), 37 in the validation set (median age, 61 years [IQR, 55–68 years]; 30 male), 75 in the internal testing set (median age, 60 years [IQR, 51–67 years]; 45 male), and 72 in the external testing set (median age, 55 years [IQR, 49–61 years]; 38 male). In the internal testing set, the best model based on MultiRecNet (the All model, with T2-weighted imaging, ADC, Dapp, Kapp, pretreatment clinical indicators, and postoperative pathologic indicators) achieved a DSC of 0.72 for tumor segmentation, an AUC of 0.97 (95% CI: 0.92, >.99) for recurrence or metastasis classification at 3 years, and a C-index of 0.92 for DFS prediction. In the external testing set, the model continued to perform well for survival prediction (C-index = 0.81, P < .001).

The MultiRecNet-based model enabled prognostic prediction in a fully automated end-to-end manner in patients with LARC following nCRT.

Keywords: MR-Imaging, Abdomen/GI, Rectum, Oncology

Supplemental material is available for this article.

Published under a CC BY 4.0 license.