Automatic Facial Paralysis Estimation with Facial Action Units

TL;DR

This paper introduces ALGRNet, a novel model integrating facial Action Units detection for automatic estimation of facial paralysis severity, demonstrating promising results on benchmark and medical datasets.

Contribution

The paper presents a new adaptive local-global relational network for facial AU detection, improving facial paralysis severity estimation accuracy.

Findings

ALGRNet achieves high AU detection accuracy on BP4D and DISFA datasets.

ALGRNet effectively estimates facial paralysis severity from facial images.

The model outperforms existing methods in AU detection and paralysis estimation.

Abstract

Facial palsy is unilateral facial nerve weakness or paralysis of rapid onset with unknown causes. Automatically estimating facial palsy severeness can be helpful for the diagnosis and treatment of people suffering from it across the world. In this work, we develop and experiment with a novel model for estimating facial palsy severity. For this, an effective Facial Action Units (AU) detection technique is incorporated into our model, where AUs refer to a unique set of facial muscle movements used to describe almost every anatomically possible facial expression. In this paper, we propose a novel Adaptive Local-Global Relational Network (ALGRNet) for facial AU detection and use it to classify facial paralysis severity. ALGRNet mainly consists of three main novel structures: (i) an adaptive region learning module that learns the adaptive muscle regions based on the detected landmarks; (ii) a skip-BiLSTM that models the latent relationships among local AUs; and (iii) a feature fusion&refining module that investigates the complementary between the local and global face. Quantitative results on two AU benchmarks, i.e., BP4D and DISFA, demonstrate our ALGRNet can achieve promising AU detection accuracy. We further demonstrate the effectiveness of its application to facial paralysis estimation by migrating ALGRNet to a facial paralysis dataset collected and annotated by medical professionals.