Borrowing or Codeswitching? Annotating for Finer-Grained Distinctions in Language Mixing

TL;DR

This paper introduces a new Twitter corpus annotated for Spanish-English codeswitching and borrowing, with clear distinctions and baseline modeling, facilitating finer-grained linguistic analysis.

Contribution

It provides a novel, well-defined corpus with token-level annotations distinguishing codeswitching from borrowing, enabling more precise linguistic and computational studies.

Findings

Corpus contains 9,500 tweets with detailed annotations.

Baseline Transformer models achieve measurable performance on label prediction.

Clear annotation guidelines improve the distinction between codeswitching and borrowing.

Abstract

We present a new corpus of Twitter data annotated for codeswitching and borrowing between Spanish and English. The corpus contains 9,500 tweets annotated at the token level with codeswitches, borrowings, and named entities. This corpus differs from prior corpora of codeswitching in that we attempt to clearly define and annotate the boundary between codeswitching and borrowing and do not treat common "internet-speak" ('lol', etc.) as codeswitching when used in an otherwise monolingual context. The result is a corpus that enables the study and modeling of Spanish-English borrowing and codeswitching on Twitter in one dataset. We present baseline scores for modeling the labels of this corpus using Transformer-based language models. The annotation itself is released with a CC BY 4.0 license, while the text it applies to is distributed in compliance with the Twitter terms of service.