Face Behavior a la carte: Expressions, Affect and Action Units in a Single Network
Dimitrios Kollias, Viktoriia Sharmanska, Stefanos Zafeiriou

TL;DR
This paper introduces FaceBehaviorNet, a multi-task neural network trained on large-scale datasets to jointly recognize facial expressions, emotions, and action units, outperforming single-task models and enabling zero- and few-shot learning for related tasks.
Contribution
It presents the first large-scale joint training of facial behavior tasks in a single network, demonstrating improved performance and versatile feature learning.
Findings
Joint training outperforms single-task models.
Proposed coupling strategies enhance training effectiveness.
Features learned enable zero- and few-shot recognition of new tasks.
Abstract
Automatic facial behavior analysis has a long history of studies in the intersection of computer vision, physiology and psychology. However it is only recently, with the collection of large-scale datasets and powerful machine learning methods such as deep neural networks, that automatic facial behavior analysis started to thrive. Three of its iconic tasks are automatic recognition of basic expressions (e.g. happy, sad, surprised), estimation of continuous emotions (e.g., valence and arousal), and detection of facial action units (activations of e.g. upper/inner eyebrows, nose wrinkles). Up until now these tasks have been mostly studied independently collecting a dataset for the task. We present the first and the largest study of all facial behaviour tasks learned jointly in a single multi-task, multi-domain and multi-label network, which we call FaceBehaviorNet. For this we utilize all…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsFace recognition and analysis · Emotion and Mood Recognition · Face Recognition and Perception
