BAPLe: Backdoor Attacks on Medical Foundational Models using Prompt Learning

TL;DR

This paper introduces BAPLe, a novel backdoor attack method on medical foundation models using prompt learning, demonstrating high success rates with minimal data, highlighting security vulnerabilities in medical AI systems.

Contribution

The work presents a new backdoor attack technique that embeds triggers during prompt learning, requiring only limited data, unlike traditional methods that need extensive retraining.

Findings

BAPLe achieves high backdoor success rates across multiple models and datasets.

The method outperforms baseline backdoor attack techniques.

It demonstrates the vulnerability of medical foundation models to prompt-based backdoor attacks.

Abstract

Medical foundation models are gaining prominence in the medical community for their ability to derive general representations from extensive collections of medical image-text pairs. Recent research indicates that these models are susceptible to backdoor attacks, which allow them to classify clean images accurately but fail when specific triggers are introduced. However, traditional backdoor attacks necessitate a considerable amount of additional data to maliciously pre-train a model. This requirement is often impractical in medical imaging applications due to the usual scarcity of data. Inspired by the latest developments in learnable prompts, this work introduces a method to embed a backdoor into the medical foundation model during the prompt learning phase. By incorporating learnable prompts within the text encoder and introducing imperceptible learnable noise trigger to the input images, we exploit the full capabilities of the medical foundation models (Med-FM). Our method, BAPLe, requires only a minimal subset of data to adjust the noise trigger and the text prompts for downstream tasks, enabling the creation of an effective backdoor attack. Through extensive experiments with four medical foundation models, each pre-trained on different modalities and evaluated across six downstream datasets, we demonstrate the efficacy of our approach. BAPLe achieves a high backdoor success rate across all models and datasets, outperforming the baseline backdoor attack methods. Our work highlights the vulnerability of Med-FMs towards backdoor attacks and strives to promote the safe adoption of Med-FMs before their deployment in real-world applications. Code is available at https://asif-hanif.github.io/baple/.