When Algorithms Play Favorites: Lookism in the Generation and Perception of Faces

TL;DR

This study investigates algorithmic lookism in facial generation and classification, revealing biases where attractiveness influences perceived traits and error rates, raising fairness concerns in digital identity systems.

Contribution

It uncovers biases in AI-generated faces and gender classifiers related to attractiveness and race, highlighting fairness issues in digital face analysis.

Findings

T2I systems associate attractiveness with positive traits

Gender classifiers have higher error rates on less-attractive faces

Biases are more pronounced for non-White women

Abstract

This paper examines how synthetically generated faces and machine learning-based gender classification algorithms are affected by algorithmic lookism, the preferential treatment based on appearance. In experiments with 13,200 synthetically generated faces, we find that: (1) text-to-image (T2I) systems tend to associate facial attractiveness to unrelated positive traits like intelligence and trustworthiness; and (2) gender classification models exhibit higher error rates on "less-attractive" faces, especially among non-White women. These result raise fairness concerns regarding digital identity systems.