Unconstrained Facial Landmark Localization with Backbone-Branches Fully-Convolutional Networks

TL;DR

This paper introduces BB-FCN, a novel fully-convolutional neural network architecture that rapidly and accurately localizes facial landmarks in unconstrained environments without pre-processing or sliding windows.

Contribution

The paper proposes a backbone-branches FCN architecture for facial landmark detection that operates efficiently and effectively in unconstrained settings, outperforming existing methods.

Findings

Achieves superior accuracy in unconstrained environments

Operates without pre-processing or sliding window methods

Offers efficient learning and inference due to architecture design

Abstract

This paper investigates how to rapidly and accurately localize facial landmarks in unconstrained, cluttered environments rather than in the well segmented face images. We present a novel Backbone-Branches Fully-Convolutional Neural Network (BB-FCN), which produces facial landmark response maps directly from raw images without relying on pre-process or sliding window approaches. BB-FCN contains one backbone and a number of network branches with each corresponding to one landmark type, and it operates in a progressive manner. Specifically, the backbone roughly detects the locations of facial landmarks by taking the whole image as input, and the branches further refine the localizations based on a local observation from the backbone's intermediate feature map. Moreover, our backbone-branches architecture does not contain full-connection layers for location regression, leading to efficient learning and inference. Our extensive experiments show that our model achieves superior performances over other state-of-the-arts under both the constrained (i.e. with face regions) and the "in the wild" scenarios.