Blind Spots and Biases: Exploring the Role of Annotator Cognitive Biases in NLP

TL;DR

This paper reviews how annotator cognitive biases influence NLP system biases, highlighting the importance of understanding human decision-making in reducing societal disparities in AI.

Contribution

It provides a comprehensive review of methodologies and ongoing research on annotation biases and their impact on NLP system fairness.

Findings

Annotator biases significantly affect NLP model outcomes.

Understanding cognitive biases can help mitigate bias propagation.

Current methodologies are evolving to address annotation-related biases.

Abstract

With the rapid proliferation of artificial intelligence, there is growing concern over its potential to exacerbate existing biases and societal disparities and introduce novel ones. This issue has prompted widespread attention from academia, policymakers, industry, and civil society. While evidence suggests that integrating human perspectives can mitigate bias-related issues in AI systems, it also introduces challenges associated with cognitive biases inherent in human decision-making. Our research focuses on reviewing existing methodologies and ongoing investigations aimed at understanding annotation attributes that contribute to bias.