Agentic AI Process Observability: Discovering Behavioral Variability

TL;DR

This paper introduces methods for observing and understanding behavioral variability in AI agents using LLMs, process discovery, and static analysis to improve debugging and control.

Contribution

It presents a novel approach combining process discovery and static analysis to enhance observability of agent behavior variability in LLM-based systems.

Findings

Process and causal discovery reveal behavioral patterns.

LLM-based static analysis distinguishes intended from unintended variability.

Instrumentation improves developer control and debugging.

Abstract

AI agents that leverage Large Language Models (LLMs) are increasingly becoming core building blocks of modern software systems. A wide range of frameworks is now available to support the specification of such applications. These frameworks enable the definition of agent setups using natural language prompting, which specifies the roles, goals, and tools assigned to the various agents involved. Within such setups, agent behavior is non-deterministic for any given input, highlighting the critical need for robust debugging and observability tools. In this work, we explore the use of process and causal discovery applied to agent execution trajectories as a means of enhancing developer observability. This approach aids in monitoring and understanding the emergent variability in agent behavior. Additionally, we complement this with LLM-based static analysis techniques to distinguish between intended and unintended behavioral variability. We argue that such instrumentation is essential for giving developers greater control over evolving specifications and for identifying aspects of functionality that may require more precise and explicit definitions.