TriCEGAR: A Trace-Driven Abstraction Mechanism for Agentic AI

TL;DR

TriCEGAR introduces an automated, trace-driven abstraction mechanism for agentic AI, enabling more reliable verification and anomaly detection without manual state abstraction design.

Contribution

It automates state abstraction from execution logs and supports online MDP construction for probabilistic verification in agentic AI systems.

Findings

Automates state abstraction from traces.

Supports online probabilistic model checking.

Enables anomaly detection through run likelihoods.

Abstract

Agentic AI systems act through tools and evolve their behavior over long, stochastic interaction traces. This setting complicates assurance, because behavior depends on nondeterministic environments and probabilistic model outputs. Prior work introduced runtime verification for agentic AI via Dynamic Probabilistic Assurance (DPA), learning an MDP online and model checking quantitative properties. A key limitation is that developers must manually define the state abstraction, which couples verification to application-specific heuristics and increases adoption friction. This paper proposes TriCEGAR, a trace-driven abstraction mechanism that automates state construction from execution logs and supports online construction of an agent behavioral MDP. TriCEGAR represents abstractions as predicate trees learned from traces and refined using counterexamples. We describe a framework-native implementation that (i) captures typed agent lifecycle events, (ii) builds abstractions from traces, (iii) constructs an MDP, and (iv) performs probabilistic model checking to compute bounds such as Pmax(success) and Pmin(failure). We also show how run likelihoods enable anomaly detection as a guardrailing signal.