Convolutional Experts Constrained Local Model for Facial Landmark Detection

TL;DR

This paper introduces a novel neural network-based local detector called Convolutional Experts Network (CEN) within a Constrained Local Model framework, significantly improving facial landmark detection accuracy, especially on challenging images.

Contribution

The paper presents a new end-to-end neural architecture, CEN, and integrates it into CE-CLM, outperforming existing methods in facial landmark detection.

Findings

Outperforms state-of-the-art baselines on four datasets.

Achieves high accuracy on challenging profile images.

Robust to variations in expression, illumination, and accessories.

Abstract

Constrained Local Models (CLMs) are a well-established family of methods for facial landmark detection. However, they have recently fallen out of favor to cascaded regression-based approaches. This is in part due to the inability of existing CLM local detectors to model the very complex individual landmark appearance that is affected by expression, illumination, facial hair, makeup, and accessories. In our work, we present a novel local detector -- Convolutional Experts Network (CEN) -- that brings together the advantages of neural architectures and mixtures of experts in an end-to-end framework. We further propose a Convolutional Experts Constrained Local Model (CE-CLM) algorithm that uses CEN as local detectors. We demonstrate that our proposed CE-CLM algorithm outperforms competitive state-of-the-art baselines for facial landmark detection by a large margin on four publicly-available datasets. Our approach is especially accurate and robust on challenging profile images.