TAI3: Testing Agent Integrity in Interpreting User Intent

TL;DR

TAI3 is a novel API-centric stress testing framework that systematically uncovers intent integrity violations in LLM agents by generating and mutating realistic tasks based on toolkit documentation.

Contribution

It introduces a semantic partitioning and datatype-aware strategy memory to improve the efficiency and effectiveness of testing LLM agents for intent violations.

Findings

Effectively uncovers intent violations in 80 toolkit APIs.

Outperforms baselines in error detection rate and query efficiency.

Generalizes well to different models and evolving APIs.

Abstract

LLM agents are increasingly deployed to automate real-world tasks by invoking APIs through natural language instructions. While powerful, they often suffer from misinterpretation of user intent, leading to the agent's actions that diverge from the user's intended goal, especially as external toolkits evolve. Traditional software testing assumes structured inputs and thus falls short in handling the ambiguity of natural language. We introduce TAI3, an API-centric stress testing framework that systematically uncovers intent integrity violations in LLM agents. Unlike prior work focused on fixed benchmarks or adversarial inputs, TAI3 generates realistic tasks based on toolkits' documentation and applies targeted mutations to expose subtle agent errors while preserving user intent. To guide testing, we propose semantic partitioning, which organizes natural language tasks into meaningful categories based on toolkit API parameters and their equivalence classes. Within each partition, seed tasks are mutated and ranked by a lightweight predictor that estimates the likelihood of triggering agent errors. To enhance efficiency, TAI3 maintains a datatype-aware strategy memory that retrieves and adapts effective mutation patterns from past cases. Experiments on 80 toolkit APIs demonstrate that TAI3 effectively uncovers intent integrity violations, significantly outperforming baselines in both error-exposing rate and query efficiency. Moreover, TAI3 generalizes well to stronger target models using smaller LLMs for test generation, and adapts to evolving APIs across domains.