A Framework for Rationale Extraction for Deep QA models

TL;DR

This paper introduces a framework using Integrated Gradients to extract rationales from deep QA models, providing insights into their decision-making process without requiring model modifications or adversarial datasets.

Contribution

It presents a novel, model-agnostic rationale extraction method for deep QA models that is easier to apply across different models and datasets.

Findings

40-80% words of extracted rationale match human rationale (precision)

6-19% of human rationale is captured in the extracted rationale (recall)

The method offers a way to interpret model decisions in reading comprehension tasks.

Abstract

As neural-network-based QA models become deeper and more complex, there is a demand for robust frameworks which can access a model's rationale for its prediction. Current techniques that provide insights on a model's working are either dependent on adversarial datasets or are proposing models with explicit explanation generation components. These techniques are time-consuming and challenging to extend to existing models and new datasets. In this work, we use `Integrated Gradients' to extract rationale for existing state-of-the-art models in the task of Reading Comprehension based Question Answering (RCQA). On detailed analysis and comparison with collected human rationales, we find that though ~40-80% words of extracted rationale coincide with the human rationale (precision), only 6-19% of human rationale is present in the extracted rationale (recall).