Towards a Deep Multi-layered Dialectal Language Analysis: A Case Study of African-American English

TL;DR

This paper explores the challenges of NLP models with African-American English, emphasizing the importance of dialectal inclusivity and proposing a human-in-the-loop approach to improve understanding and reduce bias.

Contribution

It introduces a human-in-the-loop paradigm to better understand AAE and advocates for dialectal language inclusivity in NLP systems.

Findings

NLP models trained on MAE perform poorly on AAE.

Human-in-the-loop approach enhances understanding of AAE.

Promotes dialectal inclusivity to reduce bias.

Abstract

Currently, natural language processing (NLP) models proliferate language discrimination leading to potentially harmful societal impacts as a result of biased outcomes. For example, part-of-speech taggers trained on Mainstream American English (MAE) produce non-interpretable results when applied to African American English (AAE) as a result of language features not seen during training. In this work, we incorporate a human-in-the-loop paradigm to gain a better understanding of AAE speakers' behavior and their language use, and highlight the need for dialectal language inclusivity so that native AAE speakers can extensively interact with NLP systems while reducing feelings of disenfranchisement.