Attention with Dependency Parsing Augmentation for Fine-Grained Attribution

TL;DR

This paper introduces a novel fine-grained attribution method for RAG-generated content, combining dependency parsing and attention-based similarity to improve evidence extraction and semantic completeness.

Contribution

It proposes two techniques—token-wise evidence aggregation and dependency parsing integration—that enhance attribution accuracy beyond existing model-internal similarity methods.

Findings

Outperforms prior attribution methods in experiments.

Improves semantic completeness of evidence extraction.

Enhances attribution granularity and efficiency.

Abstract

To assist humans in efficiently validating RAG-generated content, developing a fine-grained attribution mechanism that provides supporting evidence from retrieved documents for every answer span is essential. Existing fine-grained attribution methods rely on model-internal similarity metrics between responses and documents, such as saliency scores and hidden state similarity. However, these approaches suffer from either high computational complexity or coarse-grained representations. Additionally, a common problem shared by the previous works is their reliance on decoder-only Transformers, limiting their ability to incorporate contextual information after the target span. To address the above problems, we propose two techniques applicable to all model-internals-based methods. First, we aggregate token-wise evidence through set union operations, preserving the granularity of representations. Second, we enhance the attributor by integrating dependency parsing to enrich the semantic completeness of target spans. For practical implementation, our approach employs attention weights as the similarity metric. Experimental results demonstrate that the proposed method consistently outperforms all prior works.