Server, server in the cloud. Who is the fairest in the crowd?

TL;DR

This paper investigates how neural networks classify facial attractiveness, revealing biases and challenges in automated beauty assessments, and discusses implications for cultural perceptions and machine learning fairness.

Contribution

The study provides experimental analysis of CNN architectures on attractiveness classification and explores bias sources in data and model design.

Findings

Neural networks struggle with robust attractiveness detection.

Biases can originate from data and architecture choices.

Platform-level ML systems influence human feature judgments.

Abstract

This paper follows the recent history of automated beauty competitions to discuss how machine learning techniques, in particular neural networks, alter the way attractiveness is handled and how this impacts the cultural landscape. We describe experiments performed to probe the behavior of two different convolutional neural network architectures in the classification of facial attractiveness in a large database of celebrity faces. As opposed to other easily definable facial features, attractiveness is difficult to detect robustly even for the best classification systems. Based on the observations from these experiments, we discuss several approaches to detect factors that come into play when a machine evaluates human features, and how bias can occur not only in data selection but in network architectures; in multiple forms on multiple levels throughout the process. The overall goal is to map out with mixed methods a novel condition: slippages produced by platform level machine learning systems that make judgements in domains considered dependent on high level human intelligence.