Face to Purchase: Predicting Consumer Choices with Structured Facial and Behavioral Traits Embedding

TL;DR

This paper introduces a semi-supervised hierarchical embedding model that leverages facial features and purchasing history to predict consumer buying behavior, demonstrating improved accuracy on real-world data.

Contribution

It is the first to explore facial data for consumer purchase prediction, integrating facial features with behavioral data in a novel semi-supervised framework.

Findings

Facial features significantly improve purchase prediction accuracy.

The hierarchical embedding network effectively captures high-level consumer traits.

Model outperforms baseline methods on real-world dataset.

Abstract

Predicting consumers' purchasing behaviors is critical for targeted advertisement and sales promotion in e-commerce. Human faces are an invaluable source of information for gaining insights into consumer personality and behavioral traits. However, consumer's faces are largely unexplored in previous research, and the existing face-related studies focus on high-level features such as personality traits while neglecting the business significance of learning from facial data. We propose to predict consumers' purchases based on their facial features and purchasing histories. We design a semi-supervised model based on a hierarchical embedding network to extract high-level features of consumers and to predict the top-$N$ purchase destinations of a consumer. Our experimental results on a real-world dataset demonstrate the positive effect of incorporating facial information in predicting consumers' purchasing behaviors.