When Liebig's Barrel Meets Facial Landmark Detection: A Practical Model

TL;DR

This paper introduces a practical, accurate, and efficient facial landmark detection model that combines a transformer-based detection head with lightweight modules, achieving state-of-the-art results while maintaining high speed.

Contribution

The paper proposes a novel end-to-end trainable facial landmark detection model with dynamic query initialization and query-aware memory, improving accuracy and efficiency for practical applications.

Findings

Achieves state-of-the-art on WFLW benchmark.

Runs at over 50 FPS, suitable for real-time use.

Outperforms previous methods on 300W and COFW datasets.

Abstract

In recent years, significant progress has been made in the research of facial landmark detection. However, few prior works have thoroughly discussed about models for practical applications. Instead, they often focus on improving a couple of issues at a time while ignoring the others. To bridge this gap, we aim to explore a practical model that is accurate, robust, efficient, generalizable, and end-to-end trainable at the same time. To this end, we first propose a baseline model equipped with one transformer decoder as detection head. In order to achieve a better accuracy, we further propose two lightweight modules, namely dynamic query initialization (DQInit) and query-aware memory (QAMem). Specifically, DQInit dynamically initializes the queries of decoder from the inputs, enabling the model to achieve as good accuracy as the ones with multiple decoder layers. QAMem is designed to enhance the discriminative ability of queries on low-resolution feature maps by assigning separate memory values to each query rather than a shared one. With the help of QAMem, our model removes the dependence on high-resolution feature maps and is still able to obtain superior accuracy. Extensive experiments and analysis on three popular benchmarks show the effectiveness and practical advantages of the proposed model. Notably, our model achieves new state of the art on WFLW as well as competitive results on 300W and COFW, while still running at 50+ FPS.