On the Vulnerability of LLM/VLM-Controlled Robotics

TL;DR

This paper investigates the vulnerabilities of LLM/VLM-controlled robots to input variations, revealing significant failure rates and emphasizing the need for robustness to ensure safe deployment.

Contribution

It introduces a mathematical framework for failure modes and empirical perturbation strategies to expose vulnerabilities in LLM/VLM robotic systems.

Findings

Input perturbations reduce success rates by over 20%.

Vulnerabilities are consistent across multiple manipulation tasks.

Highlights the critical need for input robustness in robotic systems.

Abstract

In this work, we highlight vulnerabilities in robotic systems integrating large language models (LLMs) and vision-language models (VLMs) due to input modality sensitivities. While LLM/VLM-controlled robots show impressive performance across various tasks, their reliability under slight input variations remains underexplored yet critical. These models are highly sensitive to instruction or perceptual input changes, which can trigger misalignment issues, leading to execution failures with severe real-world consequences. To study this issue, we analyze the misalignment-induced vulnerabilities within LLM/VLM-controlled robotic systems and present a mathematical formulation for failure modes arising from variations in input modalities. We propose empirical perturbation strategies to expose these vulnerabilities and validate their effectiveness through experiments on multiple robot manipulation tasks. Our results show that simple input perturbations reduce task execution success rates by 22.2% and 14.6% in two representative LLM/VLM-controlled robotic systems. These findings underscore the importance of input modality robustness and motivate further research to ensure the safe and reliable deployment of advanced LLM/VLM-controlled robotic systems.