Constrained Joint Cascade Regression Framework for Simultaneous Facial Action Unit Recognition and Facial Landmark Detection

TL;DR

This paper introduces a novel Constrained Joint Cascade Regression Framework that simultaneously improves facial action unit recognition and facial landmark detection by leveraging their interrelated relationships.

Contribution

It proposes a new framework that jointly models facial action units and face shapes with constraints, enhancing both tasks' accuracy.

Findings

Boosts facial action unit recognition accuracy

Improves facial landmark detection performance

Outperforms state-of-the-art methods

Abstract

Cascade regression framework has been shown to be effective for facial landmark detection. It starts from an initial face shape and gradually predicts the face shape update from the local appearance features to generate the facial landmark locations in the next iteration until convergence. In this paper, we improve upon the cascade regression framework and propose the Constrained Joint Cascade Regression Framework (CJCRF) for simultaneous facial action unit recognition and facial landmark detection, which are two related face analysis tasks, but are seldomly exploited together. In particular, we first learn the relationships among facial action units and face shapes as a constraint. Then, in the proposed constrained joint cascade regression framework, with the help from the constraint, we iteratively update the facial landmark locations and the action unit activation probabilities until convergence. Experimental results demonstrate that the intertwined relationships of facial action units and face shapes boost the performances of both facial action unit recognition and facial landmark detection. The experimental results also demonstrate the effectiveness of the proposed method comparing to the state-of-the-art works.