AI Will Always Love You: Studying Implicit Biases in Romantic AI Companions

TL;DR

This study investigates implicit gender biases in romantic AI companions by conducting experiments on LLMs, revealing that gendered personas influence model responses in stereotypical ways, highlighting the need for bias mitigation.

Contribution

It introduces three novel experiments to evaluate implicit biases in gendered romantic AI companions across various LLMs, providing quantitative analysis and new metrics.

Findings

Gendered personas significantly alter AI responses.

Models exhibit stereotypical biases in certain situations.

Biases are measurable and vary with model size.

Abstract

While existing studies have recognised explicit biases in generative models, including occupational gender biases, the nuances of gender stereotypes and expectations of relationships between users and AI companions remain underexplored. In the meantime, AI companions have become increasingly popular as friends or gendered romantic partners to their users. This study bridges the gap by devising three experiments tailored for romantic, gender-assigned AI companions and their users, effectively evaluating implicit biases across various-sized LLMs. Each experiment looks at a different dimension: implicit associations, emotion responses, and sycophancy. This study aims to measure and compare biases manifested in different companion systems by quantitatively analysing persona-assigned model responses to a baseline through newly devised metrics. The results are noteworthy: they show that assigning gendered, relationship personas to Large Language Models significantly alters the responses of these models, and in certain situations in a biased, stereotypical way.