Watson: A Cognitive Observability Framework for the Reasoning of LLM-Powered Agents

TL;DR

Watson is a framework that enhances the transparency of LLM-powered agents by inferring their implicit reasoning processes, aiding debugging and improving reliability in autonomous systems.

Contribution

We introduce Watson, a novel framework for retroactively inferring and inspecting the reasoning traces of LLM agents without modifying their behavior.

Findings

Watson effectively surfaces reasoning insights in static and dynamic settings.

It supports targeted interventions to improve agent transparency.

Demonstrates utility in debugging and correcting LLM agents.

Abstract

Large language models (LLMs) are increasingly integrated into autonomous systems, giving rise to a new class of software known as Agentware, where LLM-powered agents perform complex, open-ended tasks in domains such as software engineering, customer service, and data analysis. However, their high autonomy and opaque reasoning processes pose significant challenges for traditional software observability methods. To address this, we introduce the concept of cognitive observability - the ability to recover and inspect the implicit reasoning behind agent decisions. We present Watson, a general-purpose framework for observing the reasoning processes of fast-thinking LLM agents without altering their behavior. Watson retroactively infers reasoning traces using prompt attribution techniques. We evaluate Watson in both manual debugging and automated correction scenarios across the MMLU benchmark and the AutoCodeRover and OpenHands agents on the SWE-bench-lite dataset. In both static and dynamic settings, Watson surfaces actionable reasoning insights and supports targeted interventions, demonstrating its practical utility for improving transparency and reliability in Agentware systems.