Computational Imaging Meets LLMs: Zero-Shot IDH Mutation Prediction in Brain Gliomas

TL;DR

This paper introduces a novel framework that combines large language models with computational imaging to non-invasively predict IDH mutation status in brain gliomas using multi-parametric MRI data, achieving high accuracy without manual annotations.

Contribution

It presents a zero-shot prediction framework integrating LLMs with image analytics for tumor genotyping, eliminating the need for fine-tuning or manual annotations.

Findings

High accuracy in IDH mutation prediction across multiple datasets

GPT 5 outperforms GPT 4o in phenotype interpretation

Volumetric features are the most important predictors

Abstract

We present a framework that combines Large Language Models with computational image analytics for non-invasive, zero-shot prediction of IDH mutation status in brain gliomas. For each subject, coregistered multi-parametric MRI scans and multi-class tumor segmentation maps were processed to extract interpretable semantic (visual) attributes and quantitative features, serialized in a standardized JSON file, and used to query GPT 4o and GPT 5 without fine-tuning. We evaluated this framework on six publicly available datasets (N = 1427) and results showcased high accuracy and balanced classification performance across heterogeneous cohorts, even in the absence of manual annotations. GPT 5 outperformed GPT 4o in context-driven phenotype interpretation. Volumetric features emerged as the most important predictors, supplemented by subtype-specific imaging markers and clinical information. Our results demonstrate the potential of integrating LLM-based reasoning with computational image analytics for precise, non-invasive tumor genotyping, advancing diagnostic strategies in neuro-oncology. The code is available at https://github.com/ATPLab-LUMS/CIM-LLM.