QuASE: Question-Answer Driven Sentence Encoding

TL;DR

This paper introduces QuASE, a framework that leverages QA data to learn sentence representations, improving performance on various NLP tasks without solely relying on further pre-training of language models.

Contribution

Proposes the QuASE framework that uses QA data to learn versatile sentence encodings applicable to multiple downstream NLP tasks, offering an alternative supervision method.

Findings

QA-based sentence encodings improve downstream task performance.

Distinguishes between single- and multi-sentence encoding needs.

Effective as a plugin for various NLP applications.

Abstract

Question-answering (QA) data often encodes essential information in many facets. This paper studies a natural question: Can we get supervision from QA data for other tasks (typically, non-QA ones)? For example, {\em can we use QAMR (Michael et al., 2017) to improve named entity recognition?} We suggest that simply further pre-training BERT is often not the best option, and propose the {\em question-answer driven sentence encoding (QuASE)} framework. QuASE learns representations from QA data, using BERT or other state-of-the-art contextual language models. In particular, we observe the need to distinguish between two types of sentence encodings, depending on whether the target task is a single- or multi-sentence input; in both cases, the resulting encoding is shown to be an easy-to-use plugin for many downstream tasks. This work may point out an alternative way to supervise NLP tasks.