MOON: A Mixed Objective Optimization Network for the Recognition of Facial Attributes

TL;DR

This paper introduces MOON, a novel deep learning framework that improves facial attribute recognition by jointly optimizing multiple tasks with a domain adaptive re-weighting strategy, addressing data imbalance issues.

Contribution

The paper proposes MOON, a mixed objective optimization network with a new loss function that effectively handles multi-label imbalance in facial attribute recognition.

Findings

MOON achieves state-of-the-art accuracy in facial attribute recognition.

Balanced training with MOON improves face recognition performance on LFW.

Joint multi-task optimization outperforms independent task training.

Abstract

Attribute recognition, particularly facial, extracts many labels for each image. While some multi-task vision problems can be decomposed into separate tasks and stages, e.g., training independent models for each task, for a growing set of problems joint optimization across all tasks has been shown to improve performance. We show that for deep convolutional neural network (DCNN) facial attribute extraction, multi-task optimization is better. Unfortunately, it can be difficult to apply joint optimization to DCNNs when training data is imbalanced, and re-balancing multi-label data directly is structurally infeasible, since adding/removing data to balance one label will change the sampling of the other labels. This paper addresses the multi-label imbalance problem by introducing a novel mixed objective optimization network (MOON) with a loss function that mixes multiple task objectives with domain adaptive re-weighting of propagated loss. Experiments demonstrate that not only does MOON advance the state of the art in facial attribute recognition, but it also outperforms independently trained DCNNs using the same data. When using facial attributes for the LFW face recognition task, we show that our balanced (domain adapted) network outperforms the unbalanced trained network.