A PCA based Keypoint Tracking Approach to Automated Facial Expressions Encoding

TL;DR

This paper introduces an unsupervised PCA-based method for automatically generating facial action units from keypoints, achieving high variance explanation and potential for real-time facial expression analysis.

Contribution

It presents a novel PCA-based approach to automate facial action unit generation, reducing manual effort and enabling real-time analysis.

Findings

PCA AUs explain over 92.83% of variance in datasets.

PCA AUs are comparable to FACS AUs in variance explained.

Method generalizes well across multiple datasets.

Abstract

The Facial Action Coding System (FACS) for studying facial expressions is manual and requires significant effort and expertise. This paper explores the use of automated techniques to generate Action Units (AUs) for studying facial expressions. We propose an unsupervised approach based on Principal Component Analysis (PCA) and facial keypoint tracking to generate data-driven AUs called PCA AUs using the publicly available DISFA dataset. The PCA AUs comply with the direction of facial muscle movements and are capable of explaining over 92.83 percent of the variance in other public test datasets (BP4D-Spontaneous and CK+), indicating their capability to generalize facial expressions. The PCA AUs are also comparable to a keypoint-based equivalence of FACS AUs in terms of variance explained on the test datasets. In conclusion, our research demonstrates the potential of automated techniques to be an alternative to manual FACS labeling which could lead to efficient real-time analysis of facial expressions in psychology and related fields. To promote further research, we have made code repository publicly available.