# When AI joins the table: evaluating large language model performance in soft tissue sarcoma tumor board decisions

## TL;DR

This study evaluates how well ChatGPT-4o can support tumor board decisions for soft tissue sarcoma, finding it performs well in some areas but needs expert oversight.

## Contribution

The novel contribution is a direct evaluation of ChatGPT-4o's performance in generating treatment recommendations for soft tissue sarcoma compared to expert tumor board decisions.

## Key findings

- ChatGPT-4o scores were significantly lower than the maximum achievable value across all five evaluation domains.
- Clinical contextualization was the strongest domain for ChatGPT-4o compared to other criteria.
- No significant performance differences were observed across different sarcoma subtypes.

## Abstract

Multidisciplinary tumor boards (MDTs) are critical for the personalized management of soft tissue sarcomas (STS), but they are limited by time, costs, and resource demands. With recent advances in large language models (LLMs) like ChatGPT, there is growing interest in evaluating their potential role in augmenting MDT workflows. This study aimed to assess the clinical performance of ChatGPT-4o in real-world STS cases using predefined evaluation criteria, comparing its treatment suggestions with expert MDT decisions.

This retrospective study included 152 patients presented to the multidisciplinary sarcoma tumor board. ChatGPT-4o was prompted to generate guideline-based treatment recommendations based on anonymized tumor board registration letters. Outputs were scored by blinded expert reviewers using a five-domain framework: diagnostic modalities, therapeutic modalities, treatment sequencing/timing, chemotherapy regimen, and clinical contextualization. Descriptive statistics and non-parametric ANOVA with post hoc tests assessed performance, including subgroup analysis by sarcoma subtype.

ChatGPT-4o scores were significantly lower than the maximum achievable value of 1.0 across all five criteria (all p < 0.0001). Among individual domains, clinical contextualization significantly outperformed all other criteria in pairwise comparisons (all p < 0.05). No significant performance differences were observed across sarcoma subtypes (H = 19.74, p = 0.138).

ChatGPT-4o demonstrated substantial expert-rated performance in generating tumor board recommendations for soft tissue sarcoma cases, particularly excelling in personalized contextualization. Discrepancies in treatment sequencing and chemotherapy selection highlight the need for expert oversight. These findings support the feasibility of LLM integration into oncology workflows, warranting further refinement toward safe, supportive clinical use.

The online version contains supplementary material available at 10.1007/s00432-026-06432-w.