Dynamic Survival Prediction using Longitudinal Images based on Transformer

TL;DR

This paper presents SurLonFormer, a Transformer-based model that integrates longitudinal medical images and structured data for improved survival prediction, effectively handling censored data and enhancing interpretability.

Contribution

Introduction of SurLonFormer, a novel Transformer architecture that combines imaging and structured data for survival analysis, addressing current methodological limitations.

Findings

SurLonFormer outperforms existing models in predictive accuracy.

The model effectively incorporates censored data.

It identifies meaningful imaging biomarkers for disease prognosis.

Abstract

Survival analysis utilizing multiple longitudinal medical images plays a pivotal role in the early detection and prognosis of diseases by providing insight beyond single-image evaluations. However, current methodologies often inadequately utilize censored data, overlook correlations among longitudinal images measured over multiple time points, and lack interpretability. We introduce SurLonFormer, a novel Transformer-based neural network that integrates longitudinal medical imaging with structured data for survival prediction. Our architecture comprises three key components: a Vision Encoder for extracting spatial features, a Sequence Encoder for aggregating temporal information, and a Survival Encoder based on the Cox proportional hazards model. This framework effectively incorporates censored data, addresses scalability issues, and enhances interpretability through occlusion sensitivity analysis and dynamic survival prediction. Extensive simulations and a real-world application in Alzheimer's disease analysis demonstrate that SurLonFormer achieves superior predictive performance and successfully identifies disease-related imaging biomarkers.