Once Upon a Team: Investigating Bias in LLM-Driven Software Team Composition and Task Allocation

TL;DR

This study examines how large language models (LLMs) can exhibit bias in software team composition and task allocation, revealing systematic disparities influenced by demographic attributes and stereotypes, which can exacerbate inequities.

Contribution

It provides the first comprehensive analysis of bias in LLM-driven team decisions considering combined demographic factors, highlighting the need for fairness-aware approaches.

Findings

Demographic attributes significantly influence team selection and task assignment.

LLMs reproduce stereotypes, leading to uneven distribution of roles.

Bias persists even after accounting for expertise-related factors.

Abstract

LLMs are increasingly used to boost productivity and support software engineering tasks. However, when applied to socially sensitive decisions such as team composition and task allocation, they raise concerns of fairness. Prior studies have revealed that LLMs may reproduce stereotypes; however, these analyses remain exploratory and examine sensitive attributes in isolation. This study investigates whether LLMs exhibit bias in team composition and task assignment by analyzing the combined effects of candidates' country and pronouns. Using three LLMs and 3,000 simulated decisions, we find systematic disparities: demographic attributes significantly shaped both selection likelihood and task allocation, even when accounting for expertise-related factors. Task distributions further reflected stereotypes, with technical and leadership roles unevenly assigned across groups. Our findings indicate that LLMs exacerbate demographic inequities in software engineering contexts, underscoring the need for fairness-aware assessment.