How to Synthesize a Large-Scale and Trainable Micro-Expression Dataset?

TL;DR

This paper presents a novel data synthesis protocol and dataset for micro-expression recognition, enabling large-scale training and improved model performance without technical novelty.

Contribution

It introduces a new protocol for automatically synthesizing large-scale micro-expression data and a corresponding dataset, MiE-X, facilitating better recognition models.

Findings

Synthesized MiE data improves recognition accuracy.

Discovered three types of Action Units for MiE synthesis.

MiEs generalize across different faces and relate to early macro-expressions.

Abstract

This paper does not contain technical novelty but introduces our key discoveries in a data generation protocol, a database and insights. We aim to address the lack of large-scale datasets in micro-expression (MiE) recognition due to the prohibitive cost of data collection, which renders large-scale training less feasible. To this end, we develop a protocol to automatically synthesize large scale MiE training data that allow us to train improved recognition models for real-world test data. Specifically, we discover three types of Action Units (AUs) that can constitute trainable MiEs. These AUs come from real-world MiEs, early frames of macro-expression videos, and the relationship between AUs and expression categories defined by human expert knowledge. With these AUs, our protocol then employs large numbers of face images of various identities and an off-the-shelf face generator for MiE synthesis, yielding the MiE-X dataset. MiE recognition models are trained or pre-trained on MiE-X and evaluated on real-world test sets, where very competitive accuracy is obtained. Experimental results not only validate the effectiveness of the discovered AUs and MiE-X dataset but also reveal some interesting properties of MiEs: they generalize across faces, are close to early-stage macro-expressions, and can be manually defined.