Correctness is not Faithfulness in RAG Attributions

TL;DR

This paper distinguishes between citation correctness and faithfulness in retrieval-augmented generation, revealing that many attributions lack genuine reliance on cited sources, which undermines trust in model responses.

Contribution

It clarifies the concepts of correctness and faithfulness in citations, and demonstrates the prevalence of post-rationalization undermining attribution trustworthiness.

Findings

Up to 57% of citations are not genuinely faithful.

Current evaluation often overlooks citation faithfulness.

Post-rationalization is a widespread issue in attribution.

Abstract

Retrieving relevant context is a common approach to reduce hallucinations and enhance answer reliability. Explicitly citing source documents allows users to verify generated responses and increases trust. Prior work largely evaluates citation correctness - whether cited documents support the corresponding statements. But citation correctness alone is insufficient. To establish trust in attributed answers, we must examine both citation correctness and citation faithfulness. In this work, we first disentangle the notions of citation correctness and faithfulness, which have been applied inconsistently in previous studies. Faithfulness ensures that the model's reliance on cited documents is genuine, reflecting actual reference use rather than superficial alignment with prior beliefs, which we call post-rationalization. We design an experiment that reveals the prevalent issue of post-rationalization, which undermines reliable attribution and may result in misplaced trust. Our findings suggest that current attributed answers often lack citation faithfulness (up to 57 percent of the citations), highlighting the need to evaluate correctness and faithfulness for trustworthy attribution in language models.