Parallel Sentence-Level Explanation Generation for Real-World Low-Resource Scenarios

TL;DR

This paper introduces a non-autoregressive, parallel explanation generation model for low-resource scenarios, enabling faster training and inference for sentence-level explanations without relying heavily on annotated data.

Contribution

It is the first to develop a weakly supervised to unsupervised approach for sentence-level explanations and proposes a non-autoregressive model for parallel explanation generation.

Findings

Users can train classifiers 10-15 times faster with parallel explanations.

The method performs well with minimal or no annotated data.

Experiments on NLI and Spouse Prediction tasks validate effectiveness.

Abstract

In order to reveal the rationale behind model predictions, many works have exploited providing explanations in various forms. Recently, to further guarantee readability, more and more works turn to generate sentence-level human language explanations. However, current works pursuing sentence-level explanations rely heavily on annotated training data, which limits the development of interpretability to only a few tasks. As far as we know, this paper is the first to explore this problem smoothly from weak-supervised learning to unsupervised learning. Besides, we also notice the high latency of autoregressive sentence-level explanation generation, which leads to asynchronous interpretability after prediction. Therefore, we propose a non-autoregressive interpretable model to facilitate parallel explanation generation and simultaneous prediction. Through extensive experiments on Natural Language Inference task and Spouse Prediction task, we find that users are able to train classifiers with comparable performance $10-15\times$ faster with parallel explanation generation using only a few or no annotated training data.