Grow-and-Clip: Informative-yet-Concise Evidence Distillation for Answer Explanation

TL;DR

This paper introduces Grow-and-Clip Evidence Distillation (GCED), a novel method for extracting concise, informative, and readable evidence to improve interpretability of QA models, validated through extensive experiments and human evaluation.

Contribution

The paper defines evidence as supporting facts that are informative, concise, and readable, and proposes the GCED algorithm to effectively extract such evidence for better interpretability of QA models.

Findings

GCED improves interpretability of QA answers

Automatic evidence distillation achieves human-like quality

Experimental results on SQuAD and TriviaQA datasets validate effectiveness

Abstract

Interpreting the predictions of existing Question Answering (QA) models is critical to many real-world intelligent applications, such as QA systems for healthcare, education, and finance. However, existing QA models lack interpretability and provide no feedback or explanation for end-users to help them understand why a specific prediction is the answer to a question. In this research, we argue that the evidences of an answer is critical to enhancing the interpretability of QA models. Unlike previous research that simply extracts several sentence(s) in the context as evidence, we are the first to explicitly define the concept of evidence as the supporting facts in a context which are informative, concise, and readable. Besides, we provide effective strategies to quantitatively measure the informativeness, conciseness and readability of evidence. Furthermore, we propose Grow-and-Clip Evidence Distillation (GCED) algorithm to extract evidences from the contexts by trade-off informativeness, conciseness, and readability. We conduct extensive experiments on the SQuAD and TriviaQA datasets with several baseline models to evaluate the effect of GCED on interpreting answers to questions. Human evaluation are also carried out to check the quality of distilled evidences. Experimental results show that automatic distilled evidences have human-like informativeness, conciseness and readability, which can enhance the interpretability of the answers to questions.