Subpixel Heatmap Regression for Facial Landmark Localization

TL;DR

This paper introduces a novel continuous distribution-based heatmap encoding-decoding method for facial landmark localization, significantly reducing discretization errors and achieving state-of-the-art accuracy.

Contribution

It proposes a new heatmap encoding-decoding approach leveraging continuous distributions and a Siamese training scheme for improved facial landmark localization.

Findings

Achieves state-of-the-art results on multiple datasets.

Reduces discretization errors in heatmap regression.

Enhances facial landmark localization robustness.

Abstract

Deep Learning models based on heatmap regression have revolutionized the task of facial landmark localization with existing models working robustly under large poses, non-uniform illumination and shadows, occlusions and self-occlusions, low resolution and blur. However, despite their wide adoption, heatmap regression approaches suffer from discretization-induced errors related to both the heatmap encoding and decoding process. In this work we show that these errors have a surprisingly large negative impact on facial alignment accuracy. To alleviate this problem, we propose a new approach for the heatmap encoding and decoding process by leveraging the underlying continuous distribution. To take full advantage of the newly proposed encoding-decoding mechanism, we also introduce a Siamese-based training that enforces heatmap consistency across various geometric image transformations. Our approach offers noticeable gains across multiple datasets setting a new state-of-the-art result in facial landmark localization. Code alongside the pretrained models will be made available at https://www.adrianbulat.com/face-alignment