Explain and Predict, and then Predict Again

TL;DR

This paper introduces ExPred, a multi-task learning approach that enhances explain-then-predict models by effectively integrating rationale data, leading to improved performance across various language understanding tasks.

Contribution

ExPred is a novel method that combines explanation generation with a second prediction step, effectively utilizing rationale data to boost task accuracy.

Findings

Outperforms existing explain-then-predict models on multiple datasets

Improves task performance by better integrating rationale data

Demonstrates effectiveness across fact verification, sentiment analysis, and QA

Abstract

A desirable property of learning systems is to be both effective and interpretable. Towards this goal, recent models have been proposed that first generate an extractive explanation from the input text and then generate a prediction on just the explanation called explain-then-predict models. These models primarily consider the task input as a supervision signal in learning an extractive explanation and do not effectively integrate rationales data as an additional inductive bias to improve task performance. We propose a novel yet simple approach ExPred, that uses multi-task learning in the explanation generation phase effectively trading-off explanation and prediction losses. And then we use another prediction network on just the extracted explanations for optimizing the task performance. We conduct an extensive evaluation of our approach on three diverse language datasets -- fact verification, sentiment classification, and QA -- and find that we substantially outperform existing approaches.