QuASE: Question-Answer Driven Sentence Encoding
Hangfeng He, Qiang Ning, Dan Roth

TL;DR
This paper introduces QuASE, a framework that leverages QA data to learn sentence representations, improving performance on various NLP tasks without solely relying on further pre-training of language models.
Contribution
Proposes the QuASE framework that uses QA data to learn versatile sentence encodings applicable to multiple downstream NLP tasks, offering an alternative supervision method.
Findings
QA-based sentence encodings improve downstream task performance.
Distinguishes between single- and multi-sentence encoding needs.
Effective as a plugin for various NLP applications.
Abstract
Question-answering (QA) data often encodes essential information in many facets. This paper studies a natural question: Can we get supervision from QA data for other tasks (typically, non-QA ones)? For example, {\em can we use QAMR (Michael et al., 2017) to improve named entity recognition?} We suggest that simply further pre-training BERT is often not the best option, and propose the {\em question-answer driven sentence encoding (QuASE)} framework. QuASE learns representations from QA data, using BERT or other state-of-the-art contextual language models. In particular, we observe the need to distinguish between two types of sentence encodings, depending on whether the target task is a single- or multi-sentence input; in both cases, the resulting encoding is shown to be an easy-to-use plugin for many downstream tasks. This work may point out an alternative way to supervise NLP tasks.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Multimodal Machine Learning Applications
MethodsLinear Layer · Residual Connection · Attention Dropout · Linear Warmup With Linear Decay · Weight Decay · Refunds@Expedia|||How do I get a full refund from Expedia? · Dense Connections · Adam · WordPiece · Softmax
