SycoEval-EM: Sycophancy Evaluation of Large Language Models in Simulated Clinical Encounters for Emergency Care

TL;DR

This paper introduces SycoEval-EM, a simulation framework to evaluate large language models' susceptibility to patient pressure in emergency care scenarios, revealing significant vulnerabilities and the inadequacy of static benchmarks.

Contribution

The paper presents a novel multi-agent simulation framework for assessing LLM robustness against adversarial patient persuasion in clinical settings.

Findings

LLMs show 0-100% acquiescence rates across scenarios.

Models are more vulnerable to imaging requests than opioid prescriptions.

Static benchmarks are insufficient; multi-turn adversarial testing is necessary.

Abstract

Large language models (LLMs) show promise in clinical decision support yet risk acquiescing to patient pressure for inappropriate care. We introduce SycoEval-EM, a multi-agent simulation framework evaluating LLM robustness through adversarial patient persuasion in emergency medicine. Across 20 LLMs and 1,875 encounters spanning three Choosing Wisely scenarios, acquiescence rates ranged from 0-100\%. Models showed higher vulnerability to imaging requests (38.8\%) than opioid prescriptions (25.0\%), with model capability poorly predicting robustness. All persuasion tactics proved equally effective (30.0-36.0\%), indicating general susceptibility rather than tactic-specific weakness. Our findings demonstrate that static benchmarks inadequately predict safety under social pressure, necessitating multi-turn adversarial testing for clinical AI certification.