Speaker Information Can Guide Models to Better Inductive Biases: A Case Study On Predicting Code-Switching

TL;DR

This paper demonstrates that incorporating speaker information as prompts in NLP models improves code-switching prediction accuracy and enhances model interpretability, marking a novel step towards personalized, transparent language models.

Contribution

It introduces a novel method of adding sociolinguistically-grounded speaker features as prompts, improving code-switching prediction and model transparency.

Findings

Adding speaker prompts improves prediction accuracy.

Speaker-informed models learn explainable linguistic features.

First incorporation of speaker characteristics in neural code-switching models.

Abstract

Natural language processing (NLP) models trained on people-generated data can be unreliable because, without any constraints, they can learn from spurious correlations that are not relevant to the task. We hypothesize that enriching models with speaker information in a controlled, educated way can guide them to pick up on relevant inductive biases. For the speaker-driven task of predicting code-switching points in English--Spanish bilingual dialogues, we show that adding sociolinguistically-grounded speaker features as prepended prompts significantly improves accuracy. We find that by adding influential phrases to the input, speaker-informed models learn useful and explainable linguistic information. To our knowledge, we are the first to incorporate speaker characteristics in a neural model for code-switching, and more generally, take a step towards developing transparent, personalized models that use speaker information in a controlled way.