Learning Correct Behavior from Examples: Validating Sequential Execution in Autonomous Agents

TL;DR

This paper introduces a novel algorithm that learns correct sequential behavior of autonomous agents from minimal examples and validates new executions using semantic understanding and topological matching.

Contribution

It combines dominator analysis with large language model semantics to automatically learn and validate agent behavior from very few execution traces.

Findings

High accuracy in detecting bugs with only 3 training traces

Effective across domains like UI testing, code generation, and robotics

Provides explainable validation with coverage metrics

Abstract

As autonomous agents become increasingly sophisticated, validating their sequential behavior presents a significant challenge. Traditional testing approaches require manual specification, exact sequence matching, or thousands of training examples. We present a novel algorithm that automatically learns correct behavior from just 2-10 passing execution traces and validates new executions against this learned model. Our approach combines dominator analysis from compiler theory with multimodal large language model-powered semantic understanding to identify essential states and handle non-deterministic behavior. The system constructs a generalized ground truth model using Prefix Tree Acceptors, merges traces through multi-tiered equivalence detection, and validates new executions via topological subsequence matching. In controlled experiments, our system achieved high accuracy in detecting product bugs and false successes using only 3 training traces. This approach provides explainable validation results with coverage metrics and works across diverse domains including UI testing, code generation, and robotic processes.