# Evaluating the Clinical Decision-Making Accuracy of Artificial Intelligence in Common Geriatric Syndromes Using Evidence-Based Guidelines

## TL;DR

This study evaluates how well ChatGPT can help with geriatric care decisions using standard medical scenarios and finds it has strengths but also notable limitations.

## Contribution

The study is one of the first to assess AI clinical decision-making in geriatrics using expert ratings and standardized vignettes.

## Key findings

- ChatGPT scored highest in clarity and safety but lower in accuracy and completeness.
- Advance care planning had the highest scores, while urinary incontinence had the lowest.
- Key omissions in responses included missing assessments and guideline-recommended tools.

## Abstract

Background

Artificial intelligence (AI) tools such as ChatGPT are increasingly being explored for clinical decision support, yet their role in geriatric medicine remains uncertain due to the complexity of multimorbidity and care planning. This study aimed to evaluate the clinical accuracy, completeness, and guideline alignment of ChatGPT’s responses to common geriatric scenarios using standardized vignettes.

Methodology

Seven standardized vignettes representing common geriatric scenarios, namely, polypharmacy, falls, dementia, delirium, frailty, advance care planning, and urinary incontinence, were submitted to ChatGPT (GPT-5). Responses were evaluated by five independent consultant geriatricians using a standardized rubric across the following five domains: accuracy, completeness, guideline alignment, safety, and clarity (0-2 score per domain). Descriptive statistics summarized performance, and qualitative feedback was thematically analyzed. Inter-rater reliability was assessed using Krippendorff’s alpha.

Results

ChatGPT scored the highest in clarity (66/70) and safety (63/70), with slightly lower performance in accuracy (59/70) and completeness (55/70). Guideline alignment was generally strong (61/70). Advance care planning received the highest domain scores; urinary incontinence scored the lowest. Krippendorff’s alpha showed high inter-rater agreement (0.969). Reviewers identified key omissions, such as missing assessments or guideline-recommended tools, in multiple vignettes.

Conclusions

ChatGPT showed potential as a supportive tool in geriatric care, offering clear and generally safe responses aligned with guidelines. However, it lacked clinical depth and missed key elements in complex scenarios. AI tools such as ChatGPT should be used with caution, under expert oversight, and not as standalone decision makers in clinical practice.