Interpretable Survival Prediction for Colorectal Cancer using Deep Learning

TL;DR

This study develops a deep learning system for predicting colorectal cancer survival that is interpretable, revealing both known and novel histologic features associated with prognosis, and demonstrating improved predictive accuracy over traditional clinicopathologic features.

Contribution

The paper introduces a method to interpret deep learning prognostic models by identifying human-interpretable histologic features that explain model variance and enhance understanding of prognosis.

Findings

Deep learning model achieved 0.70 and 0.69 AUC for 5-year survival prediction.

Histologic features explain 73-80% of the variance in model scores.

Identified a novel prognostic feature with high accuracy and visual distinctiveness.

Abstract

Deriving interpretable prognostic features from deep-learning-based prognostic histopathology models remains a challenge. In this study, we developed a deep learning system (DLS) for predicting disease specific survival for stage II and III colorectal cancer using 3,652 cases (27,300 slides). When evaluated on two validation datasets containing 1,239 cases (9,340 slides) and 738 cases (7,140 slides) respectively, the DLS achieved a 5-year disease-specific survival AUC of 0.70 (95%CI 0.66-0.73) and 0.69 (95%CI 0.64-0.72), and added significant predictive value to a set of 9 clinicopathologic features. To interpret the DLS, we explored the ability of different human-interpretable features to explain the variance in DLS scores. We observed that clinicopathologic features such as T-category, N-category, and grade explained a small fraction of the variance in DLS scores (R2=18% in both validation sets). Next, we generated human-interpretable histologic features by clustering embeddings from a deep-learning based image-similarity model and showed that they explain the majority of the variance (R2 of 73% to 80%). Furthermore, the clustering-derived feature most strongly associated with high DLS scores was also highly prognostic in isolation. With a distinct visual appearance (poorly differentiated tumor cell clusters adjacent to adipose tissue), this feature was identified by annotators with 87.0-95.5% accuracy. Our approach can be used to explain predictions from a prognostic deep learning model and uncover potentially-novel prognostic features that can be reliably identified by people for future validation studies.