Moving from 2D to 3D: volumetric medical image classification for rectal cancer staging

TL;DR

This paper introduces a volumetric CNN model that leverages 3D MRI data to improve the accuracy of preoperative rectal cancer staging, outperforming radiologists in classifying T2 versus T3 stages.

Contribution

The study presents a novel 3D CNN architecture with a custom ResNet encoder, bilinear feature aggregation, and combined loss functions for improved rectal cancer stage classification.

Findings

Achieved an AUC of 0.831, surpassing radiologist accuracy.

Demonstrated the effectiveness of 3D convolutional models in medical volume analysis.

Validated the model's potential for extension to other volumetric medical tasks.

Abstract

Volumetric images from Magnetic Resonance Imaging (MRI) provide invaluable information in preoperative staging of rectal cancer. Above all, accurate preoperative discrimination between T2 and T3 stages is arguably both the most challenging and clinically significant task for rectal cancer treatment, as chemo-radiotherapy is usually recommended to patients with T3 (or greater) stage cancer. In this study, we present a volumetric convolutional neural network to accurately discriminate T2 from T3 stage rectal cancer with rectal MR volumes. Specifically, we propose 1) a custom ResNet-based volume encoder that models the inter-slice relationship with late fusion (i.e., 3D convolution at the last layer), 2) a bilinear computation that aggregates the resulting features from the encoder to create a volume-wise feature, and 3) a joint minimization of triplet loss and focal loss. With MR volumes of pathologically confirmed T2/T3 rectal cancer, we perform extensive experiments to compare various designs within the framework of residual learning. As a result, our network achieves an AUC of 0.831, which is higher than the reported accuracy of the professional radiologist groups. We believe this method can be extended to other volume analysis tasks