Post-Hoc Answer Attribution for Grounded and Trustworthy Long Document Comprehension: Task, Insights, and Challenges

TL;DR

This paper introduces a new task of post-hoc answer attribution for long document comprehension, highlighting dataset limitations and evaluating existing systems' strengths and weaknesses.

Contribution

It formulates the answer attribution task for long documents and refactors datasets to evaluate current retrieval and entailment-based systems.

Findings

Existing datasets have limitations for this task

Current systems show strengths and weaknesses in attribution

Need for better datasets to assess system performance

Abstract

Attributing answer text to its source document for information-seeking questions is crucial for building trustworthy, reliable, and accountable systems. We formulate a new task of post-hoc answer attribution for long document comprehension (LDC). Owing to the lack of long-form abstractive and information-seeking LDC datasets, we refactor existing datasets to assess the strengths and weaknesses of existing retrieval-based and proposed answer decomposition and textual entailment-based optimal selection attribution systems for this task. We throw light on the limitations of existing datasets and the need for datasets to assess the actual performance of systems on this task.