A Structured Literature Review on Traditional Approaches in Current Natural Language Processing

TL;DR

This paper reviews the current state of traditional NLP techniques amidst the dominance of neural models, highlighting their continued relevance and specific roles in various NLP application scenarios.

Contribution

It provides a comprehensive survey of traditional NLP approaches across five key application areas, analyzing their usage and relevance in the era of large language models.

Findings

Traditional models are still actively used in NLP applications.

Traditional approaches serve as baselines or components within modern pipelines.

All five surveyed scenarios retain some form of traditional techniques.

Abstract

The continued rise of neural networks and large language models in the more recent past has altered the natural language processing landscape, enabling new approaches towards typical language tasks and achieving mainstream success. Despite the huge success of large language models, many disadvantages still remain and through this work we assess the state of the art in five application scenarios with a particular focus on the future perspectives and sensible application scenarios of traditional and older approaches and techniques. In this paper we survey recent publications in the application scenarios classification, information and relation extraction, text simplification as well as text summarization. After defining our terminology, i.e., which features are characteristic for traditional techniques in our interpretation for the five scenarios, we survey if such traditional approaches are still being used, and if so, in what way they are used. It turns out that all five application scenarios still exhibit traditional models in one way or another, as part of a processing pipeline, as a comparison/baseline to the core model of the respective paper, or as the main model(s) of the paper. For the complete statistics, see https://zenodo.org/records/13683801