Evaluating and Correcting Human Annotation Bias in Dynamic Micro-Expression Recognition

TL;DR

This paper introduces GAMDSS, a novel method for improving micro-expression recognition by dynamically selecting key frames to reduce human annotation bias, especially across different cultures.

Contribution

The paper proposes GAMDSS, a new architecture that enhances spatio-temporal modeling of micro-expressions by dynamically reselecting key frames, reducing annotation errors and improving recognition accuracy.

Findings

GAMDSS effectively reduces human annotation bias in multicultural datasets.

Offset-frame annotations are more uncertain, highlighting the need for standardized labeling.

The method can be integrated into existing models without increasing parameters.

Abstract

Existing manual labeling of micro-expressions is subject to errors in accuracy, especially in cross-cultural scenarios where deviation in labeling of key frames is more prominent. To address this issue, this paper presents a novel Global Anti-Monotonic Differential Selection Strategy (GAMDSS) architecture for enhancing the effectiveness of spatio-temporal modeling of micro-expressions through keyframe re-selection. Specifically, the method identifies Onset and Apex frames, which are characterized by significant micro-expression variation, from complete micro-expression action sequences via a dynamic frame reselection mechanism. It then uses these to determine Offset frames and construct a rich spatio-temporal dynamic representation. A two-branch structure with shared parameters is then used to efficiently extract spatio-temporal features. Extensive experiments are conducted on seven widely recognized micro-expression datasets. The results demonstrate that GAMDSS effectively reduces subjective errors caused by human factors in multicultural datasets such as SAMM and 4DME. Furthermore, quantitative analyses confirm that offset-frame annotations in multicultural datasets are more uncertain, providing theoretical justification for standardizing micro-expression annotations. These findings directly support our argument for reconsidering the validity and generalizability of dataset annotation paradigms. Notably, this design can be integrated into existing models without increasing the number of parameters, offering a new approach to enhancing micro-expression recognition performance. The source code is available on GitHub[https://github.com/Cross-Innovation-Lab/GAMDSS].