Exploring Implicit Perspectives on Autism in Large Language Models Through Multi-Agent Simulations

TL;DR

This paper investigates implicit biases about autism in large language models like ChatGPT using multi-agent simulations, revealing assumptions about social dependence and proposing ways to improve model interactions with autistic individuals.

Contribution

It introduces a novel multi-agent simulation approach to analyze LLM biases regarding autism and suggests integrating the double empathy problem to enhance model understanding and communication.

Findings

ChatGPT assumes autistic people are socially dependent.

Multi-agent simulations reveal biases in LLMs about autism.

Proposes incorporating the double empathy problem to improve LLM interactions.

Abstract

Large Language Models (LLMs) like ChatGPT offer potential support for autistic people, but this potential requires understanding the implicit perspectives these models might carry, including their biases and assumptions about autism. Moving beyond single-agent prompting, we utilized LLM-based multi-agent systems to investigate complex social scenarios involving autistic and non-autistic agents. In our study, agents engaged in group-task conversations and answered structured interview questions, which we analyzed to examine ChatGPT's biases and how it conceptualizes autism. We found that ChatGPT assumes autistic people are socially dependent, which may affect how it interacts with autistic users or conveys information about autism. To address these challenges, we propose adopting the double empathy problem, which reframes communication breakdowns as a mutual challenge. We describe how future LLMs could address the biases we observed and improve interactions involving autistic people by incorporating the double empathy problem into their design.