AegisAgent: An Autonomous Defense Agent Against Prompt Injection Attacks in LLM-HARs

TL;DR

This paper presents AegisAgent, an autonomous defense system for LLM-based human activity recognition, which actively detects and mitigates prompt injection attacks to improve security and trustworthiness.

Contribution

It introduces AegisAgent, a novel autonomous agent that actively perceives, reasons, and acts to defend against prompt injection attacks in LLM-driven HAR systems.

Findings

Reduces attack success rate by 30% on average

Incurs only 78.6 ms latency overhead on GPU

Effective against 15 common prompt injection attacks

Abstract

The integration of Large Language Models (LLMs) into wearable sensing is creating a new class of mobile applications capable of nuanced human activity understanding. However, the reliability of these systems is critically undermined by their vulnerability to prompt injection attacks, where attackers deliberately input deceptive instructions into LLMs. Traditional defenses, based on static filters and rigid rules, are insufficient to address the semantic complexity of these new attacks. We argue that a paradigm shift is needed -- from passive filtering to active protection and autonomous reasoning. We introduce AegisAgent, an autonomous agent system designed to ensure the security of LLM-driven HAR systems. Instead of merely blocking threats, AegisAgent functions as a cognitive guardian. It autonomously perceives potential semantic inconsistencies, reasons about the user's true intent by consulting a dynamic memory of past interactions, and acts by generating and executing a multi-step verification and repair plan. We implement AegisAgent as a lightweight, full-stack prototype and conduct a systematic evaluation on 15 common attacks with five state-of-the-art LLM-based HAR systems on three public datasets. Results show it reduces attack success rate by 30\% on average while incurring only 78.6 ms of latency overhead on a GPU workstation. Our work makes the first step towards building secure and trustworthy LLM-driven HAR systems.