Heterogeneous Face Attribute Estimation: A Deep Multi-Task Learning Approach

TL;DR

This paper introduces a deep multi-task learning approach that jointly estimates multiple heterogeneous face attributes, explicitly modeling attribute correlation and heterogeneity, resulting in superior performance across various benchmarks.

Contribution

It presents a novel CNN-based multi-task learning framework that handles attribute heterogeneity and correlation, and introduces an extended face database with demographic attributes.

Findings

Outperforms state-of-the-art methods on multiple benchmarks.

Effectively models attribute correlation and heterogeneity.

Demonstrates strong generalization on a public face database.

Abstract

Face attribute estimation has many potential applications in video surveillance, face retrieval, and social media. While a number of methods have been proposed for face attribute estimation, most of them did not explicitly consider the attribute correlation and heterogeneity (e.g., ordinal vs. nominal and holistic vs. local) during feature representation learning. In this paper, we present a Deep Multi-Task Learning (DMTL) approach to jointly estimate multiple heterogeneous attributes from a single face image. In DMTL, we tackle attribute correlation and heterogeneity with convolutional neural networks (CNNs) consisting of shared feature learning for all the attributes, and category-specific feature learning for heterogeneous attributes. We also introduce an unconstrained face database (LFW+), an extension of public-domain LFW, with heterogeneous demographic attributes (age, gender, and race) obtained via crowdsourcing. Experimental results on benchmarks with multiple face attributes (MORPH II, LFW+, CelebA, LFWA, and FotW) show that the proposed approach has superior performance compared to state of the art. Finally, evaluations on a public-domain face database (LAP) with a single attribute show that the proposed approach has excellent generalization ability.