Self-supervised Auxiliary Learning for Texture and Model-based Hybrid Robust and Fair Featuring in Face Analysis

TL;DR

This paper introduces a self-supervised auxiliary learning approach using mask auto-encoders to enhance texture feature representation, improving robustness and fairness in face analysis tasks like attribute recognition, emotion detection, and deepfake identification.

Contribution

The novel integration of self-supervised mask auto-encoder tasks as auxiliary learning enhances feature robustness and fairness in face analysis models.

Findings

Improved feature representation for fair face analysis.

Enhanced robustness against biases in face tasks.

Effective application across multiple face analysis paradigms.

Abstract

In this work, we explore Self-supervised Learning (SSL) as an auxiliary task to blend the texture-based local descriptors into feature modelling for efficient face analysis. Combining a primary task and a self-supervised auxiliary task is beneficial for robust representation. Therefore, we used the SSL task of mask auto-encoder (MAE) as an auxiliary task to reconstruct texture features such as local patterns along with the primary task for robust and unbiased face analysis. We experimented with our hypothesis on three major paradigms of face analysis: face attribute and face-based emotion analysis, and deepfake detection. Our experiment results exhibit that better feature representation can be gleaned from our proposed model for fair and bias-less face analysis.