Subjective Face Transform using Human First Impressions

TL;DR

This paper introduces a generative model-based framework to modify faces along subjective first impression attributes while preserving identity, aiding understanding of biases and improving predictive models.

Contribution

It presents an end-to-end method for semantically meaningful, identity-preserving face edits along subjective attributes, unlike prior statistical feature manipulations.

Findings

Effective in changing perceived attributes without losing identity.

Demonstrates generalizability to real and synthetic faces.

Improves first impression prediction models with augmented data.

Abstract

Humans tend to form quick subjective first impressions of non-physical attributes when seeing someone's face, such as perceived trustworthiness or attractiveness. To understand what variations in a face lead to different subjective impressions, this work uses generative models to find semantically meaningful edits to a face image that change perceived attributes. Unlike prior work that relied on statistical manipulation in feature space, our end-to-end framework considers trade-offs between preserving identity and changing perceptual attributes. It maps latent space directions to changes in attribute scores, enabling a perceptually significant identity-preserving transformation of any input face along an attribute axis according to a target change. We train on real and synthetic faces, evaluate for in-domain and out-of-domain images using predictive models and human ratings, demonstrating the generalizability of our approach. Ultimately, such a framework can be used to understand and explain trends and biases in subjective interpretation of faces that are not dependent on the subject's identity. This is demonstrated with improved model performance for first impression prediction when augmenting the training data with images generated by the proposed approach for a wider range of input to learn associations between face features and subjective attributes.