C$^2$-Cite: Contextual-Aware Citation Generation for Attributed Large Language Models

TL;DR

C$^2$-Cite is a framework that improves attribution in large language models by explicitly aligning citation markers with their referenced content, enhancing the integration and reliability of generated citations.

Contribution

The paper introduces C$^2$-Cite, a novel contextual-aware citation generation method that explicitly encodes and aligns citation markers with source content, addressing limitations of existing attribution techniques.

Findings

Outperforms SOTA by 5.8% in citation quality

Achieves 17.4% improvement in response correctness

Validated on ALCE benchmark across three datasets

Abstract

The attribution technique enhances the credibility of LLMs by adding citations to the generated sentences, enabling users to trace back to the original sources and verify the reliability of the output. However, existing instruction-tuned attributed LLMs often fail to properly interpret the contextual semantics of citation symbols (e.g., [i]) during text generation. This shortcoming arises from their insufficient awareness of the context information surrounding citation markers, which in turn leads to disjointed references and poor integration of retrieved knowledge into the generated content. To address this issue, we propose a novel \textbf{C}ontextual-aware \textbf{C}itation generation framework (\textbf{C$^2$}-\textbf{Cite}) that explicitly integrates the semantic relationships between citation markers and their referenced content. Specifically, a contextual citation alignment mechanism is adopted: it first encodes the retrieved document contexts into the symbol representation of citations, then aligns the marker numbers by decoding information from a citation router function. This mechanism enables the transformation of citation markers from generic placeholders into active knowledge pointers that link to the referenced source information. Experimental results on the ALCE benchmark across three datasets validate our framework C$^2$-Cite++: it outperforms the SOTA baseline by an average of 5.8\% in citation quality and 17.4\% in response correctness. The implementation is publicly available at https://github.com/BAI-LAB/c2cite