Shape Preserving Facial Landmarks with Graph Attention Networks

TL;DR

This paper introduces a novel facial landmark estimation model combining CNNs with Graph Attention Networks, effectively capturing facial structure and improving accuracy especially under challenging appearance variations.

Contribution

It proposes a new model that integrates appearance and location encoding with attention mechanisms and a multi-task, coarse-to-fine approach for improved facial landmark estimation.

Findings

Achieves top performance on head pose and landmark benchmarks.

Effectively models global facial structure.

Most beneficial in cases of large local appearance changes.

Abstract

Top-performing landmark estimation algorithms are based on exploiting the excellent ability of large convolutional neural networks (CNNs) to represent local appearance. However, it is well known that they can only learn weak spatial relationships. To address this problem, we propose a model based on the combination of a CNN with a cascade of Graph Attention Network regressors. To this end, we introduce an encoding that jointly represents the appearance and location of facial landmarks and an attention mechanism to weigh the information according to its reliability. This is combined with a multi-task approach to initialize the location of graph nodes and a coarse-to-fine landmark description scheme. Our experiments confirm that the proposed model learns a global representation of the structure of the face, achieving top performance in popular benchmarks on head pose and landmark estimation. The improvement provided by our model is most significant in situations involving large changes in the local appearance of landmarks.