PINA: Prompt Injection Attack against Navigation Agents

TL;DR

This paper introduces PINA, a framework demonstrating high success rates of prompt injection attacks on navigation agents powered by large language models, revealing significant security vulnerabilities in embodied AI systems.

Contribution

It is the first systematic study of prompt injection attacks on navigation agents, proposing an adaptive attack framework tailored for real-world, executable navigation tasks.

Findings

Achieves an average attack success rate of 87.5%

Surpasses all baseline attack methods in effectiveness

Remains robust under various attack conditions

Abstract

Navigation agents powered by large language models (LLMs) convert natural language instructions into executable plans and actions. Compared to text-based applications, their security is far more critical: a successful prompt injection attack does not just alter outputs but can directly misguide physical navigation, leading to unsafe routes, mission failure, or real-world harm. Despite this high-stakes setting, the vulnerability of navigation agents to prompt injection remains largely unexplored. In this paper, we propose PINA, an adaptive prompt optimization framework tailored to navigation agents under black-box, long-context, and action-executable constraints. Experiments on indoor and outdoor navigation agents show that PINA achieves high attack success rates with an average ASR of 87.5%, surpasses all baselines, and remains robust under ablation and adaptive-attack conditions. This work provides the first systematic investigation of prompt injection attacks in navigation and highlights their urgent security implications for embodied LLM agents.