Semantic Uncertainty Quantification of Hallucinations in LLMs: A Quantum Tensor Network Based Method

TL;DR

This paper introduces a quantum tensor network based framework for quantifying semantic uncertainty in LLM hallucinations, enabling more reliable detection and prioritization of trustworthy outputs.

Contribution

It presents a novel quantum physics inspired method for uncertainty quantification in LLMs, improving hallucination detection and output reliability assessment.

Findings

Consistent AUROC and AURAC improvements over baselines

Robustness across different generation lengths and quantization levels

Effective in resource-constrained deployment scenarios

Abstract

Large language models (LLMs) exhibit strong generative capabilities but remain vulnerable to confabulations, fluent yet unreliable outputs that vary arbitrarily even under identical prompts. Leveraging a quantum tensor network based pipeline, we propose a quantum physics inspired uncertainty quantification framework that accounts for aleatoric uncertainty in token sequence probability for semantic equivalence based clustering of LLM generations. This offers a principled and interpretable scheme for hallucination detection. We further introduce an entropy maximization strategy that prioritizes high certainty, semantically coherent outputs and highlights entropy regions where LLM decisions are likely to be unreliable, offering practical guidelines for when human oversight is warranted. We evaluate the robustness of our scheme under different generation lengths and quantization levels, dimensions overlooked in prior studies, demonstrating that our approach remains reliable even in resource constrained deployments. A total of 116 experiments on TriviaQA, NQ, SVAMP, and SQuAD across multiple architectures including Mistral-7B, Mistral-7B-instruct, Falcon-rw-1b, LLaMA-3.2-1b, LLaMA-2-13b-chat, LLaMA-2-7b-chat, LLaMA-2-13b, and LLaMA-2-7b show consistent improvements in AUROC and AURAC over state of the art baselines.