Comparing Humans and Models on a Similar Scale: Towards Cognitive Gender Bias Evaluation in Coreference Resolution

TL;DR

This paper investigates how model biases in coreference resolution compare to human biases by using dual-process theory and crowdsourcing experiments, revealing that humans are slightly more biased on real data, but models are more biased on synthetic data.

Contribution

It introduces a novel framework to quantify and compare human and model gender biases in NLP using dual-process theory and experimental methods.

Findings

Humans are ~3% more biased on real data.

Models are ~12% more biased on synthetic data.

The dual-process approach offers new insights into bias mechanisms.

Abstract

Spurious correlations were found to be an important factor explaining model performance in various NLP tasks (e.g., gender or racial artifacts), often considered to be ''shortcuts'' to the actual task. However, humans tend to similarly make quick (and sometimes wrong) predictions based on societal and cognitive presuppositions. In this work we address the question: can we quantify the extent to which model biases reflect human behaviour? Answering this question will help shed light on model performance and provide meaningful comparisons against humans. We approach this question through the lens of the dual-process theory for human decision-making. This theory differentiates between an automatic unconscious (and sometimes biased) ''fast system'' and a ''slow system'', which when triggered may revisit earlier automatic reactions. We make several observations from two crowdsourcing experiments of gender bias in coreference resolution, using self-paced reading to study the ''fast'' system, and question answering to study the ''slow'' system under a constrained time setting. On real-world data humans make $\sim$3\% more gender-biased decisions compared to models, while on synthetic data models are $\sim$12\% more biased.