Learning Preference-Based Similarities from Face Images using Siamese Multi-Task CNNs

TL;DR

This paper explores whether deep learning models can predict personal compatibility for romantic matches based solely on face images, linking visual features to preferences and attitudes.

Contribution

It introduces a Siamese Multi-Task CNN approach to predict similarity scores from face images, connecting visual cues with personal preferences for the first time.

Findings

Significant correlation between predicted and actual similarity scores.

Feasibility of inferring preferences from face images demonstrated.

Potential for improving visual-based matching in online dating.

Abstract

Online dating has become a common occurrence over the last few decades. A key challenge for online dating platforms is to determine suitable matches for their users. A lot of dating services rely on self-reported user traits and preferences for matching. At the same time, some services largely rely on user images and thus initial visual preference. Especially for the latter approach, previous research has attempted to capture users' visual preferences for automatic match recommendation. These approaches are mostly based on the assumption that physical attraction is the key factor for relationship formation and personal preferences, interests, and attitude are largely neglected. Deep learning approaches have shown that a variety of properties can be predicted from human faces to some degree, including age, health and even personality traits. Therefore, we investigate the feasibility of bridging image-based matching and matching with personal interests, preferences, and attitude. We approach the problem in a supervised manner by predicting similarity scores between two users based on images of their faces only. The ground-truth for the similarity matching scores is determined by a test that aims to capture users' preferences, interests, and attitude that are relevant for forming romantic relationships. The images are processed by a Siamese Multi-Task deep learning architecture. We find a statistically significant correlation between predicted and target similarity scores. Thus, our results indicate that learning similarities in terms of interests, preferences, and attitude from face images appears to be feasible to some degree.