Generative Discrimination: What Happens When Generative AI Exhibits Bias, and What Can Be Done About It

TL;DR

This paper examines how generative AI can produce biased or discriminatory outputs, analyzes legal challenges, and proposes updates to regulations like the EU AI Act to mitigate such issues.

Contribution

It identifies specific types of biases in genAI outputs and advocates for legal reforms and technical standards to address discrimination effectively.

Findings

GenAI systems often produce biased representations of protected groups.

Traditional legal frameworks are inadequate for addressing genAI-specific discrimination.

Proposed legal updates include mandatory testing, auditing, and bias mitigation standards.

Abstract

As generative Artificial Intelligence (genAI) technologies proliferate across sectors, they offer significant benefits but also risk exacerbating discrimination. This chapter explores how genAI intersects with non-discrimination laws, identifying shortcomings and suggesting improvements. It highlights two main types of discriminatory outputs: (i) demeaning and abusive content and (ii) subtler biases due to inadequate representation of protected groups, which may not be overtly discriminatory in individual cases but have cumulative discriminatory effects. For example, genAI systems may predominantly depict white men when asked for images of people in important jobs. This chapter examines these issues, categorizing problematic outputs into three legal categories: discriminatory content; harassment; and legally hard cases like unbalanced content, harmful stereotypes or misclassification. It argues for holding genAI providers and deployers liable for discriminatory outputs and highlights the inadequacy of traditional legal frameworks to address genAI-specific issues. The chapter suggests updating EU laws, including the AI Act, to mitigate biases in training and input data, mandating testing and auditing, and evolving legislation to enforce standards for bias mitigation and inclusivity as technology advances.