Paradigm Shift in Natural Language Processing

TL;DR

This paper reviews recent paradigm shifts in NLP driven by pre-trained language models, highlighting how reformulating tasks as different paradigms enhances performance and unifies diverse NLP tasks.

Contribution

It provides a comprehensive review of the emerging paradigm shifts in NLP, emphasizing their potential to unify tasks and improve model effectiveness.

Findings

Paradigm shifts have significantly advanced NLP performance.

Reformulating tasks as different paradigms enables better task handling.

Unified models for multiple NLP tasks are becoming feasible.

Abstract

In the era of deep learning, modeling for most NLP tasks has converged to several mainstream paradigms. For example, we usually adopt the sequence labeling paradigm to solve a bundle of tasks such as POS-tagging, NER, Chunking, and adopt the classification paradigm to solve tasks like sentiment analysis. With the rapid progress of pre-trained language models, recent years have observed a rising trend of Paradigm Shift, which is solving one NLP task by reformulating it as another one. Paradigm shift has achieved great success on many tasks, becoming a promising way to improve model performance. Moreover, some of these paradigms have shown great potential to unify a large number of NLP tasks, making it possible to build a single model to handle diverse tasks. In this paper, we review such phenomenon of paradigm shifts in recent years, highlighting several paradigms that have the potential to solve different NLP tasks.