RePFormer: Refinement Pyramid Transformer for Robust Facial Landmark Detection

TL;DR

RePFormer introduces a pyramid transformer-based approach that refines facial landmark detection by modeling landmark relations and context, achieving superior robustness and accuracy in complex scenarios.

Contribution

The paper proposes a novel Refinement Pyramid Transformer that models landmark relations and context for improved facial landmark detection.

Findings

Outperforms existing methods on four benchmarks

Demonstrates high robustness in complex real-world scenarios

Effective end-to-end landmark refinement process

Abstract

This paper presents a Refinement Pyramid Transformer (RePFormer) for robust facial landmark detection. Most facial landmark detectors focus on learning representative image features. However, these CNN-based feature representations are not robust enough to handle complex real-world scenarios due to ignoring the internal structure of landmarks, as well as the relations between landmarks and context. In this work, we formulate the facial landmark detection task as refining landmark queries along pyramid memories. Specifically, a pyramid transformer head (PTH) is introduced to build both homologous relations among landmarks and heterologous relations between landmarks and cross-scale contexts. Besides, a dynamic landmark refinement (DLR) module is designed to decompose the landmark regression into an end-to-end refinement procedure, where the dynamically aggregated queries are transformed to residual coordinates predictions. Extensive experimental results on four facial landmark detection benchmarks and their various subsets demonstrate the superior performance and high robustness of our framework.