Variation is the Norm: Embracing Sociolinguistics in NLP

TL;DR

This paper advocates for integrating sociolinguistic variation into NLP research, demonstrating how embracing language variation, exemplified through Luxembourgish, can improve model robustness and performance.

Contribution

It introduces a framework combining sociolinguistics with NLP, emphasizing the importance of including language variation in model training and evaluation.

Findings

Models perform significantly worse on data with high orthographic variation.

Including variation in fine-tuning improves NLP model robustness.

Variation-aware models show better generalization across language forms.

Abstract

In Natural Language Processing (NLP), variation is typically seen as noise and "normalised away" before processing, even though it is an integral part of language. Conversely, studying language variation in social contexts is central to sociolinguistics. We present a framework to combine the sociolinguistic dimension of language with the technical dimension of NLP. We argue that by embracing sociolinguistics, variation can actively be included in a research setup, in turn informing the NLP side. To illustrate this, we provide a case study on Luxembourgish, an evolving language featuring a large amount of orthographic variation, demonstrating how NLP performance is impacted. The results show large discrepancies in the performance of models tested and fine-tuned on data with a large amount of orthographic variation in comparison to data closer to the (orthographic) standard. Furthermore, we provide a possible solution to improve the performance by including variation in the fine-tuning process. This case study highlights the importance of including variation in the research setup, as models are currently not robust to occurring variation. Our framework facilitates the inclusion of variation in the thought-process while also being grounded in the theoretical framework of sociolinguistics.