BadViM: Backdoor Attack against Vision Mamba

TL;DR

This paper introduces BadViM, a novel backdoor attack framework targeting Vision Mamba models, using frequency-based triggers and internal state manipulation to achieve high success rates and resist defenses.

Contribution

The paper presents BadViM, the first backdoor attack specifically designed for Vision Mamba, employing Resonant Frequency Triggers and Hidden State Alignment to enhance stealth and effectiveness.

Findings

BadViM achieves high attack success rates.

Resilient against common defenses like PatchDrop and JPEG compression.

Maintains high accuracy on clean data.

Abstract

Vision State Space Models (SSMs), particularly architectures like Vision Mamba (ViM), have emerged as promising alternatives to Vision Transformers (ViTs). However, the security implications of this novel architecture, especially their vulnerability to backdoor attacks, remain critically underexplored. Backdoor attacks aim to embed hidden triggers into victim models, causing the model to misclassify inputs containing these triggers while maintaining normal behavior on clean inputs. This paper investigates the susceptibility of ViM to backdoor attacks by introducing BadViM, a novel backdoor attack framework specifically designed for Vision Mamba. The proposed BadViM leverages a Resonant Frequency Trigger (RFT) that exploits the frequency sensitivity patterns of the victim model to create stealthy, distributed triggers. To maximize attack efficacy, we propose a Hidden State Alignment loss that strategically manipulates the internal representations of model by aligning the hidden states of backdoor images with those of target classes. Extensive experimental results demonstrate that BadViM achieves superior attack success rates while maintaining clean data accuracy. Meanwhile, BadViM exhibits remarkable resilience against common defensive measures, including PatchDrop, PatchShuffle and JPEG compression, which typically neutralize normal backdoor attacks.