Predicting First Impressions with Deep Learning

TL;DR

This paper introduces a deep learning framework to predict subjective social attributes from facial images, correlating well with crowd ratings, and extends biometric modeling to include subjective impression assessment.

Contribution

It presents a novel CNN-based regression method for modeling subjective social attributes from facial images, addressing the challenge of lacking ground truth data.

Findings

Models show strong correlation with human crowd ratings.

Framework effectively predicts subjective social impressions.

Extends biometric analysis to subjective attribute modeling.

Abstract

Describable visual facial attributes are now commonplace in human biometrics and affective computing, with existing algorithms even reaching a sufficient point of maturity for placement into commercial products. These algorithms model objective facets of facial appearance, such as hair and eye color, expression, and aspects of the geometry of the face. A natural extension, which has not been studied to any great extent thus far, is the ability to model subjective attributes that are assigned to a face based purely on visual judgements. For instance, with just a glance, our first impression of a face may lead us to believe that a person is smart, worthy of our trust, and perhaps even our admiration - regardless of the underlying truth behind such attributes. Psychologists believe that these judgements are based on a variety of factors such as emotional states, personality traits, and other physiognomic cues. But work in this direction leads to an interesting question: how do we create models for problems where there is no ground truth, only measurable behavior? In this paper, we introduce a new convolutional neural network-based regression framework that allows us to train predictive models of crowd behavior for social attribute assignment. Over images from the AFLW face database, these models demonstrate strong correlations with human crowd ratings.