Pilot Validation of AI-Derived Features for Prognostic Models in Geriatric Oncology

TL;DR

This study shows that adding AI-extracted information from clinical notes improves survival predictions for older lung cancer patients compared to using only standard data.

Contribution

The novel contribution is demonstrating that NLP-derived features from clinical notes significantly enhance prognostic model accuracy in geriatric oncology.

Findings

The enriched model with NLP features achieved higher AUC (0.768) and C-Statistic (0.717) than the baseline model.

Functional status and chemotherapy timing from clinical notes were key predictors in the improved model.

The enriched model effectively stratified patients into distinct prognostic groups with well-calibrated predictions.

Abstract

Accurate survival prediction for older adults with lung cancer is critical for guiding supportive care, yet standard models often underutilize valuable information in clinical notes. The objective of this study was to determine if a prognostic model integrating features from unstructured clinical notes could more accurately predict survival in older adults with lung cancer than a model using structured data alone. We studied 1,391 patients (≥60 years at time of diagnosis) with lung cancer, analyzing 19,377 clinical encounters. We compared a baseline survival model using structured electronic health record (EHR) data to an enriched model that integrated previously validated natural language processing (NLP)-derived features, including functional status. Models were evaluated for 12-month mortality using AUC and C-Statistic. Thirty-eight percent of patients presented with metastatic disease. The baseline prediction model achieved good performance (AUC: 0.740; C-Statistic: 0.697). The enriched model demonstrated statistically significant improvement (AUC: 0.768 vs. 0.740; C-Statistic: 0.717 vs. 0.697; DeLong’s test p<.0001) with well-calibrated predictions and effective risk stratification into distinct prognostic groups. Feature analysis confirmed NLP-derived functional status and longitudinal chemotherapy timing were key predictors, with functional status ranking among the top three most important features. Clinical notes contain crucial prognostic information not captured in structured data. Leveraging NLP to extract this narrative data offers a powerful pathway to more accurate survival prediction, which could enable better identification of high-risk patients to improve supportive care interventions.