Disambiguation of morpho-syntactic features of African American English -- the case of habitual be

TL;DR

This paper develops a workflow combining rule-based filtering and data augmentation to improve NLP disambiguation of habitual 'be' in African American English, reducing bias and achieving a 0.65 F1 score.

Contribution

It introduces a novel approach to address linguistic feature bias in NLP for AAE by balancing training data with rule-based and augmented data.

Findings

Achieved 0.65 F1 score in disambiguating habitual 'be'

Demonstrated effectiveness of combined rule-based and data augmentation methods

Reduced bias against African American English in NLP systems

Abstract

Recent research has highlighted that natural language processing (NLP) systems exhibit a bias against African American speakers. The bias errors are often caused by poor representation of linguistic features unique to African American English (AAE), due to the relatively low probability of occurrence of many such features in training data. We present a workflow to overcome such bias in the case of habitual "be". Habitual "be" is isomorphic, and therefore ambiguous, with other forms of "be" found in both AAE and other varieties of English. This creates a clear challenge for bias in NLP technologies. To overcome the scarcity, we employ a combination of rule-based filters and data augmentation that generate a corpus balanced between habitual and non-habitual instances. With this balanced corpus, we train unbiased machine learning classifiers, as demonstrated on a corpus of AAE transcribed texts, achieving .65 F$_1$ score disambiguating habitual "be".