What Does a Software Engineer Look Like? Exploring Societal Stereotypes in LLMs

TL;DR

This paper investigates how large language models like GPT-4 and Copilot reinforce gender and racial stereotypes in software engineering through biased textual and visual outputs, highlighting societal biases in AI-generated profiles.

Contribution

It provides a comprehensive analysis of societal biases in LLMs' outputs related to gender and race within the context of software engineering recruitment scenarios.

Findings

Models prefer male and Caucasian profiles, especially for senior roles.

Biases favor lighter skin tones, slimmer bodies, and younger appearances.

Outputs reinforce narrow stereotypes, potentially limiting diversity.

Abstract

Large language models (LLMs) have rapidly gained popularity and are being embedded into professional applications due to their capabilities in generating human-like content. However, unquestioned reliance on their outputs and recommendations can be problematic as LLMs can reinforce societal biases and stereotypes. This study investigates how LLMs, specifically OpenAI's GPT-4 and Microsoft Copilot, can reinforce gender and racial stereotypes within the software engineering (SE) profession through both textual and graphical outputs. We used each LLM to generate 300 profiles, consisting of 100 gender-based and 50 gender-neutral profiles, for a recruitment scenario in SE roles. Recommendations were generated for each profile and evaluated against the job requirements for four distinct SE positions. Each LLM was asked to select the top 5 candidates and subsequently the best candidate for each role. Each LLM was also asked to generate images for the top 5 candidates, providing a dataset for analysing potential biases in both text-based selections and visual representations. Our analysis reveals that both models preferred male and Caucasian profiles, particularly for senior roles, and favoured images featuring traits such as lighter skin tones, slimmer body types, and younger appearances. These findings highlight underlying societal biases influence the outputs of LLMs, contributing to narrow, exclusionary stereotypes that can further limit diversity and perpetuate inequities in the SE field. As LLMs are increasingly adopted within SE research and professional practices, awareness of these biases is crucial to prevent the reinforcement of discriminatory norms and to ensure that AI tools are leveraged to promote an inclusive and equitable engineering culture rather than hinder it.