ComFace: Facial Representation Learning with Synthetic Data for Comparing Faces

TL;DR

ComFace introduces a synthetic-data-based facial representation learning method that effectively captures intra-personal facial changes, enabling improved comparison of faces for health and emotion monitoring without relying on real intra-personal face images.

Contribution

This work presents a novel approach using synthetic face images for intra-personal facial change detection, outperforming existing methods trained on real data.

Findings

Achieves comparable or better performance than state-of-the-art methods.

Effectively captures intra-personal facial changes using synthetic data.

Enables downstream tasks like expression, weight, and age change estimation.

Abstract

Daily monitoring of intra-personal facial changes associated with health and emotional conditions has great potential to be useful for medical, healthcare, and emotion recognition fields. However, the approach for capturing intra-personal facial changes is relatively unexplored due to the difficulty of collecting temporally changing face images. In this paper, we propose a facial representation learning method using synthetic images for comparing faces, called ComFace, which is designed to capture intra-personal facial changes. For effective representation learning, ComFace aims to acquire two feature representations, i.e., inter-personal facial differences and intra-personal facial changes. The key point of our method is the use of synthetic face images to overcome the limitations of collecting real intra-personal face images. Facial representations learned by ComFace are transferred to three extensive downstream tasks for comparing faces: estimating facial expression changes, weight changes, and age changes from two face images of the same individual. Our ComFace, trained using only synthetic data, achieves comparable to or better transfer performance than general pre-training and state-of-the-art representation learning methods trained using real images.