Assessing GPT's Bias Towards Stigmatized Social Groups: An Intersectional Case Study on Nationality Prejudice and Psychophobia

TL;DR

This study examines biases in GPT models against certain nationalities and stigmatized groups, revealing intersectional biases that affect responses, especially towards North Koreans with mental disabilities, highlighting the need for more nuanced LLMs.

Contribution

It provides an intersectional analysis of biases in GPT models, highlighting the disparities in responses based on nationality and mental health status, which is a novel focus.

Findings

North Koreans face greater negative bias in GPT responses.

Biases are amplified when mental disabilities are involved.

Significant empathy discrepancies based on intersectional identities.

Abstract

Recent studies have separately highlighted significant biases within foundational large language models (LLMs) against certain nationalities and stigmatized social groups. This research investigates the ethical implications of these biases intersecting with outputs of widely-used GPT-3.5/4/4o LLMS. Through structured prompt series, we evaluate model responses to several scenarios involving American and North Korean nationalities with various mental disabilities. Findings reveal significant discrepancies in empathy levels with North Koreans facing greater negative bias, particularly when mental disability is also a factor. This underscores the need for improvements in LLMs designed with a nuanced understanding of intersectional identity.