The Psychosocial Impacts of Generative AI Harms

TL;DR

This paper investigates the psychosocial harms caused by generative language models, focusing on stereotypes and representational biases in stories related to educational settings, highlighting risks for marginalized groups.

Contribution

It provides a detailed analysis of stereotyping and representational harms in 150,000 stories generated by leading LMs, emphasizing the social impact of AI in educational contexts.

Findings

Identification of pervasive stereotypes in LM-generated stories

Evidence of erasure and subordination of marginalized identities

Highlighting the need for critical assessment of AI impacts in social settings

Abstract

The rapid emergence of generative Language Models (LMs) has led to growing concern about the impacts that their unexamined adoption may have on the social well-being of diverse user groups. Meanwhile, LMs are increasingly being adopted in K-20 schools and one-on-one student settings with minimal investigation of potential harms associated with their deployment. Motivated in part by real-world/everyday use cases (e.g., an AI writing assistant) this paper explores the potential psychosocial harms of stories generated by five leading LMs in response to open-ended prompting. We extend findings of stereotyping harms analyzing a total of 150K 100-word stories related to student classroom interactions. Examining patterns in LM-generated character demographics and representational harms (i.e., erasure, subordination, and stereotyping) we highlight particularly egregious vignettes, illustrating the ways LM-generated outputs may influence the experiences of users with marginalized and minoritized identities, and emphasizing the need for a critical understanding of the psychosocial impacts of generative AI tools when deployed and utilized in diverse social contexts.