When Minor Edits Matter: LLM-Driven Prompt Attack for Medical VLM Robustness in Ultrasound

TL;DR

This paper reveals that medical vision-language models for ultrasound are vulnerable to small, adversarial prompt changes generated by large language models, exposing significant robustness gaps critical for safe clinical use.

Contribution

It introduces a scalable adversarial evaluation framework using LLMs to generate realistic prompt variations and systematically assesses Med-VLM robustness in ultrasound analysis.

Findings

Med-VLMs are highly sensitive to minor prompt variations.

Attacker LLM capacity influences attack success.

Identified consistent failure patterns across models.

Abstract

Ultrasound is widely used in clinical practice due to its portability, cost-effectiveness, safety, and real-time imaging capabilities. However, image acquisition and interpretation remain highly operator dependent, motivating the development of robust AI-assisted analysis methods. Vision-language models (VLMs) have recently demonstrated strong multimodal reasoning capabilities and competitive performance in medical image analysis, including ultrasound. However, emerging evidence highlights significant concerns about their trustworthiness. In particular, adversarial robustness is critical because Med-VLMs operate via natural-language instructions, rendering prompt formulation a realistic and practically exploitable point of vulnerability. Small variations (typos, shorthand, underspecified requests, or ambiguous wording) can meaningfully shift model outputs. We propose a scalable adversarial evaluation framework that leverages a large language model (LLM) to generate clinically plausible adversarial prompt variants via "humanized" rewrites and minimal edits that mimic routine clinical communication. Using ultrasound multiple-choice question answering benchmarks, we systematically assess the vulnerability of SOTA Med-VLMs to these attacks, examine how attacker LLM capacity influences attack success, analyze the relationship between attack success and model confidence, and identify consistent failure patterns across models. Our results highlight realistic robustness gaps that must be addressed for safe clinical translation. Code will be released publicly following the review process.