TRAP: Hijacking VLA CoT-Reasoning via Adversarial Patches

TL;DR

This paper reveals a vulnerability in vision-language-action models with Chain-of-Thought reasoning, demonstrating how adversarial patches can hijack robotic actions without altering user instructions, highlighting security concerns.

Contribution

The paper introduces TRAP, the first targeted adversarial attack framework that manipulates CoT reasoning in VLA models using physical patches, exposing security vulnerabilities.

Findings

TRAP effectively hijacks VLA model outputs across multiple architectures.

Physical patches printed on paper can successfully attack real-world systems.

CoT reasoning strongly influences action generation, even when misaligned with instructions.

Abstract

By integrating Chain-of-Thought(CoT) reasoning, Vision-Language-Action (VLA) models have demonstrated strong capabilities in robotic manipulation, particularly by improving generalization and interpretability. However, the security of CoT-based reasoning mechanisms remains largely unexplored. In this paper, we show that CoT reasoning introduces a novel attack vector for targeted control hijacking--for example, causing a robot to mistakenly deliver a knife to a person instead of an apple--without modifying the user's instruction. We first provide empirical evidence that CoT strongly governs action generation, even when it is semantically misaligned with the input instructions. Building on this observation, we propose TRAP, the first targeted adversarial attack framework for CoT-reasoning VLA models. TRAP uses an adversarial patch (e.g., a coaster placed on the table) to corrupt intermediate CoT reasoning and hijack the VLA's output. By optimizing the CoT adversarial loss, TRAP induces specific and adversary-defined behaviors. Extensive evaluations across 3 mainstream VLA architectures and 3 CoT reasoning paradigms validate the effectiveness of TRAP. Notably, we implemented the patch by printing it on paper in a real-world setting. Our findings highlight the urgent need to secure CoT reasoning in VLA systems.