Learning Deep Representation for Face Alignment with Auxiliary Attributes

TL;DR

This paper proposes a multi-task deep learning approach that jointly learns face alignment and auxiliary facial attribute recognition, improving robustness and reducing model complexity.

Contribution

It introduces a novel tasks-constrained deep model with dynamic task coefficients to enhance convergence and inter-task learning for face alignment.

Findings

Outperforms existing face alignment methods, especially with occlusion and pose variation.

Reduces model complexity compared to cascaded deep models.

Enhances robustness by leveraging auxiliary facial attributes.

Abstract

In this study, we show that landmark detection or face alignment task is not a single and independent problem. Instead, its robustness can be greatly improved with auxiliary information. Specifically, we jointly optimize landmark detection together with the recognition of heterogeneous but subtly correlated facial attributes, such as gender, expression, and appearance attributes. This is non-trivial since different attribute inference tasks have different learning difficulties and convergence rates. To address this problem, we formulate a novel tasks-constrained deep model, which not only learns the inter-task correlation but also employs dynamic task coefficients to facilitate the optimization convergence when learning multiple complex tasks. Extensive evaluations show that the proposed task-constrained learning (i) outperforms existing face alignment methods, especially in dealing with faces with severe occlusion and pose variation, and (ii) reduces model complexity drastically compared to the state-of-the-art methods based on cascaded deep model.