Pose-disentangled Contrastive Learning for Self-supervised Facial Representation

TL;DR

This paper introduces a pose-disentangled contrastive learning method for self-supervised facial representation, effectively capturing pose details and improving performance on multiple facial understanding tasks.

Contribution

The proposed PCL method uniquely disentangles pose features from face features using a pose-disentangled decoder and pose-related contrastive learning, enhancing facial representation learning.

Findings

Outperforms state-of-the-art SSL methods on facial tasks

Effectively disentangles pose information from face features

Improves accuracy in facial expression, recognition, AU detection, and head pose estimation

Abstract

Self-supervised facial representation has recently attracted increasing attention due to its ability to perform face understanding without relying on large-scale annotated datasets heavily. However, analytically, current contrastive-based self-supervised learning (SSL) still performs unsatisfactorily for learning facial representation. More specifically, existing contrastive learning (CL) tends to learn pose-invariant features that cannot depict the pose details of faces, compromising the learning performance. To conquer the above limitation of CL, we propose a novel Pose-disentangled Contrastive Learning (PCL) method for general self-supervised facial representation. Our PCL first devises a pose-disentangled decoder (PDD) with a delicately designed orthogonalizing regulation, which disentangles the pose-related features from the face-aware features; therefore, pose-related and other pose-unrelated facial information could be performed in individual subnetworks and do not affect each other's training. Furthermore, we introduce a pose-related contrastive learning scheme that learns pose-related information based on data augmentation of the same image, which would deliver more effective face-aware representation for various downstream tasks. We conducted linear evaluation on four challenging downstream facial understanding tasks, ie, facial expression recognition, face recognition, AU detection and head pose estimation. Experimental results demonstrate that our method significantly outperforms state-of-the-art SSL methods. Code is available at https://github.com/DreamMr/PCL}{https://github.com/DreamMr/PCL