Explaining Question Answering Models through Text Generation

TL;DR

This paper introduces a method for explaining question answering models by generating human-interpretable hypotheses, enabling better understanding of the knowledge used by large language models in answering questions.

Contribution

The authors propose a novel approach that combines hypothesis generation with classification to improve interpretability without sacrificing performance.

Findings

Model achieves comparable accuracy to end-to-end models.

Generated hypotheses provide insights into the model's reasoning.

Method enhances interpretability of question answering systems.

Abstract

Large pre-trained language models (LMs) have been shown to perform surprisingly well when fine-tuned on tasks that require commonsense and world knowledge. However, in end-to-end architectures, it is difficult to explain what is the knowledge in the LM that allows it to make a correct prediction. In this work, we propose a model for multi-choice question answering, where a LM-based generator generates a textual hypothesis that is later used by a classifier to answer the question. The hypothesis provides a window into the information used by the fine-tuned LM that can be inspected by humans. A key challenge in this setup is how to constrain the model to generate hypotheses that are meaningful to humans. We tackle this by (a) joint training with a simple similarity classifier that encourages meaningful hypotheses, and (b) by adding loss functions that encourage natural text without repetitions. We show on several tasks that our model reaches performance that is comparable to end-to-end architectures, while producing hypotheses that elucidate the knowledge used by the LM for answering the question.