Facial Landmark Correlation Analysis

TL;DR

This paper analyzes the inherent correlations among facial landmarks using Canonical Correlation Analysis, revealing strong relationships in dense annotations and enabling insights into model predictions and a new few-shot annotation method.

Contribution

It introduces a correlation analysis of facial landmarks with CCA, providing insights into model predictions and proposing a few-shot learning approach for dense annotation.

Findings

Dense facial landmarks are strongly correlated.

CNNs progressively learn facial landmarks.

Few-shot learning reduces manual annotation effort.

Abstract

We present a facial landmark position correlation analysis as well as its applications. Although numerous facial landmark detection methods have been presented in the literature, few of them explicitly take into account the inherent relationship among landmarks. To reveal and interpret this relationship, we propose to analyze landmark correlation by using Canonical Correlation Analysis~(CCA). We experimentally show that the dense facial landmark annotations in current benchmarks are strongly correlated. We propose two applications based on this analysis. First, by analyzing the landmark correlation, we gain some interesting insights into the predictions of different landmark detection models (including random forests model and CNN models). We also demonstrate how CNNs progressively learn to predict facial landmarks. Second, we propose a few-shot learning method that allows to considerably reduce the manual effort for dense landmark annotation.