Integrated Face Analytics Networks through Cross-Dataset Hybrid Training

TL;DR

This paper introduces iFAN, an integrated face analytics network that performs multiple face-related tasks simultaneously, leveraging task interactions and a novel cross-dataset training strategy to achieve state-of-the-art results.

Contribution

The paper presents a novel multi-task face analytics network with explicit task interaction modeling and a cross-dataset hybrid training method for improved performance.

Findings

Achieves 91.15% F-score on Helen face parsing dataset.

Attains 5.81% normalized mean error on MTFL landmark localization.

Reaches 45.73% accuracy on BNU emotion recognition dataset.

Abstract

Face analytics benefits many multimedia applications. It consists of a number of tasks, such as facial emotion recognition and face parsing, and most existing approaches generally treat these tasks independently, which limits their deployment in real scenarios. In this paper we propose an integrated Face Analytics Network (iFAN), which is able to perform multiple tasks jointly for face analytics with a novel carefully designed network architecture to fully facilitate the informative interaction among different tasks. The proposed integrated network explicitly models the interactions between tasks so that the correlations between tasks can be fully exploited for performance boost. In addition, to solve the bottleneck of the absence of datasets with comprehensive training data for various tasks, we propose a novel cross-dataset hybrid training strategy. It allows "plug-in and play" of multiple datasets annotated for different tasks without the requirement of a fully labeled common dataset for all the tasks. We experimentally show that the proposed iFAN achieves state-of-the-art performance on multiple face analytics tasks using a single integrated model. Specifically, iFAN achieves an overall F-score of 91.15% on the Helen dataset for face parsing, a normalized mean error of 5.81% on the MTFL dataset for facial landmark localization and an accuracy of 45.73% on the BNU dataset for emotion recognition with a single model.