Semantic Grounding Index: Geometric Bounds on Context Engagement in RAG Systems

TL;DR

The paper introduces the Semantic Grounding Index (SGI), a geometric measure in embedding space that detects when RAG system responses are semantically lazy or hallucinated, with strong empirical validation and theoretical backing.

Contribution

It proposes the SGI metric based on angular distances, providing a new, theoretically grounded tool for assessing response engagement in RAG systems.

Findings

SGI effectively detects semantic laziness in RAG responses.

SGI's discriminative power increases with question-context angular separation.

SGI scores correlate with response length and question brevity.

Abstract

When retrieval-augmented generation (RAG) systems hallucinate, what geometric trace does this leave in embedding space? We introduce the Semantic Grounding Index (SGI), defined as the ratio of angular distances from the response to the question versus the context on the unit hypersphere $\mathbb{S}^{d-1}$.Our central finding is \emph{semantic laziness}: hallucinated responses remain angularly proximate to questions rather than departing toward retrieved contexts. On HaluEval ($n$=5,000), we observe large effect sizes (Cohen's $d$ ranging from 0.92 to 1.28) across five embedding models with mean cross-model correlation $r$=0.85. Crucially, we derive from the spherical triangle inequality that SGI's discriminative power should increase with question-context angular separation $\theta(q,c)$-a theoretical prediction confirmed empirically: effect size rises monotonically from $d$=0.61 -low $\theta(q,c)$, to $d$=1.27 -high $\theta(q,c)$, with AUC improving from 0.72 to 0.83. Subgroup analysis reveals that SGI excels on long responses ($d$=2.05) and short questions ($d$=1.22), while remaining robust across context lengths. Calibration analysis yields ECE=0.10, indicating SGI scores can serve as probability estimates, not merely rankings. A critical negative result on TruthfulQA (AUC=0.478) establishes that angular geometry measures topical engagement rather than factual accuracy. SGI provides computationally efficient, theoretically grounded infrastructure for identifying responses that warrant verification in production RAG deployments.