Learning Social Relation Traits from Face Images

TL;DR

This paper presents a deep learning approach to infer fine-grained social relation traits from face images by capturing multiple facial attributes and reasoning about pairwise face relations, even with incomplete data.

Contribution

The paper introduces a novel deep model with a bridging layer that leverages multiple attribute datasets and handles missing labels for social relation prediction from face images.

Findings

Effective in predicting social relation traits from face images.

Capable of handling heterogeneous attribute data and missing labels.

Proven success in images and videos.

Abstract

Social relation defines the association, e.g, warm, friendliness, and dominance, between two or more people. Motivated by psychological studies, we investigate if such fine-grained and high-level relation traits can be characterised and quantified from face images in the wild. To address this challenging problem we propose a deep model that learns a rich face representation to capture gender, expression, head pose, and age-related attributes, and then performs pairwise-face reasoning for relation prediction. To learn from heterogeneous attribute sources, we formulate a new network architecture with a bridging layer to leverage the inherent correspondences among these datasets. It can also cope with missing target attribute labels. Extensive experiments show that our approach is effective for fine-grained social relation learning in images and videos.