Unifying Question Answering, Text Classification, and Regression via Span Extraction

TL;DR

This paper proposes a unified span extraction framework that can handle question answering, text classification, and regression tasks, simplifying model design and improving performance across various benchmarks.

Contribution

It introduces a novel unified span extraction approach that replaces task-specific output layers, enabling cross-task learning and better performance.

Findings

Unified span extraction achieves comparable or superior results.

Improves performance in low-data and multi-task settings.

Simplifies model architecture across tasks.

Abstract

Even as pre-trained language encoders such as BERT are shared across many tasks, the output layers of question answering, text classification, and regression models are significantly different. Span decoders are frequently used for question answering, fixed-class, classification layers for text classification, and similarity-scoring layers for regression tasks, We show that this distinction is not necessary and that all three can be unified as span extraction. A unified, span-extraction approach leads to superior or comparable performance in supplementary supervised pre-trained, low-data, and multi-task learning experiments on several question answering, text classification, and regression benchmarks.