Semantic Faithfulness and Entropy Production Measures to Tame Your LLM Demons and Manage Hallucinations

TL;DR

This paper introduces two unsupervised, information-theoretic metrics for evaluating the faithfulness of LLM outputs and managing hallucinations, based on thermodynamics and entropy concepts, demonstrated on SEC 10-K summarization.

Contribution

It proposes novel unsupervised metrics for LLM faithfulness and hallucination control using thermodynamics and information theory, modeling LLMs as bipartite information engines.

Findings

The semantic faithfulness metric correlates with reduced hallucinations.

High faithfulness scores are associated with low entropy production.

Framework effectively evaluates LLM summarization of SEC filings.

Abstract

Evaluating faithfulness of Large Language Models (LLMs) to a given task is a complex challenge. We propose two new unsupervised metrics for faithfulness evaluation using insights from information theory and thermodynamics. Our approach treats an LLM as a bipartite information engine where hidden layers act as a Maxwell demon controlling transformations of context $C $ into answer $A$ via prompt $Q$. We model Question-Context-Answer (QCA) triplets as probability distributions over shared topics. Topic transformations from $C$ to $Q$ and $A$ are modeled as transition matrices ${\bf Q}$ and ${\bf A}$ encoding the query goal and actual result, respectively. Our semantic faithfulness (SF) metric quantifies faithfulness for any given QCA triplet by the Kullback-Leibler (KL) divergence between these matrices. Both matrices are inferred simultaneously via convex optimization of this KL divergence, and the final SF metric is obtained by mapping the minimal divergence onto the unit interval [0,1], where higher scores indicate greater faithfulness. Furthermore, we propose a thermodynamics-based semantic entropy production (SEP) metric in answer generation, and show that high faithfulness generally implies low entropy production. The SF and SEP metrics can be used jointly or separately for LLM evaluation and hallucination control. We demonstrate our framework on LLM summarization of corporate SEC 10-K filings.