Knowledge Poisoning Attacks on Medical Multi-Modal Retrieval-Augmented Generation

TL;DR

This paper introduces M3Att, a framework for covertly poisoning medical multimodal retrieval-augmented generation systems by injecting imperceptible visual perturbations and ambiguous misinformation, degrading diagnostic accuracy.

Contribution

It presents a novel poisoning method that requires limited database knowledge and evades model correction, highlighting vulnerabilities in medical RAG systems.

Findings

M3Att consistently produces clinically plausible yet incorrect outputs.

The framework effectively manipulates retrieval probabilities with minimal visual perturbations.

It degrades diagnostic accuracy while avoiding self-correction in LLMs.

Abstract

Retrieval-augmented generation (RAG) is a widely adopted paradigm for enhancing LLMs in medical applications by incorporating expert multimodal knowledge during generation. However, the underlying retrieval databases may naturally contain, or be intentionally injected with, adversarial knowledge, which can perturb model outputs and undermine system reliability. To investigate this risk, prior studies have explored knowledge poisoning attacks in medical RAG systems. Nevertheless, most of them rely on the strong assumption that adversaries possess prior knowledge of user queries, which is unrealistic in deployments and substantially limits their practical applicability. In this paper, we propose M\textsuperscript{3}Att, a knowledge-poisoning framework designed for medical multimodal RAG systems, assuming only limited distribution knowledge of the underlying database. Our core idea is to inject covert misinformation into textual data while using paired visual data as a query-agnostic trigger to promote retrieval. We first propose a unified framework that introduces imperceptible perturbations to visual inputs to manipulate retrieval probabilities. Besides, due to the prior medical knowledge in LLMs, naively poisoned medical content with explicit factual errors can be corrected during generation. Thus, we leverage the inherent ambiguity of medical diagnosis and design a covert misinformation injection strategy that degrades diagnostic accuracy while evading model self-correction. Experiments on five LLMs and datasets demonstrate that M\textsuperscript{3}Att consistently produces clinically plausible yet incorrect generations. Codes: https://github.com/ypr17/M3Att.