Latent Debate: A Surrogate Framework for Interpreting LLM Thinking

TL;DR

This paper introduces latent debate, a framework that interprets LLM internal reasoning by capturing hidden supporting and attacking signals within a single model, aiding in understanding hallucinations and internal mechanisms.

Contribution

The paper proposes a novel, model- and task-agnostic latent debate framework that interprets LLM thinking and detects hallucinations through internal argument signals.

Findings

Latent debate aligns closely with original LLM predictions.

High latent debate activity correlates with increased hallucination risk.

Framework provides a new approach for understanding LLM internal processes.

Abstract

Understanding the internal thinking process of Large Language Models (LLMs) and the cause of hallucinations remains a key challenge. To this end, we introduce latent debate, a novel framework for interpreting model predictions through the lens of implicit internal arguments. Unlike the current work of self-consistency and multi-agent debate, which relies on explicit debates among multiple answers or multiple models, latent debate captures the hidden supporting and attacking signals that arise within a single model during a single inference. We first present a model- and task-agnostic conceptual framework, and then instantiate it symbolically to approximate the thinking process of LLMs on True/False prediction tasks. Empirical studies demonstrate that latent debate is a faithful structured surrogate model that has highly consistent predictions with the original LLM. Beyond interpretability, we demonstrate that latent debate provides a strong baseline for hallucination detection. Further analysis reveals strong correlations between hallucinations and debate patterns, such as a high degree of latent debates in the middle layers is linked to a higher risk of hallucinations. These findings position latent debate as a potential framework for understanding internal mechanisms of LLMs, especially for scenarios where internal (dis)agreements appear during the inference steps.