Region attention and graph embedding network for occlusion objective class-based micro-expression recognition

TL;DR

This paper introduces a novel region-inspired relation reasoning network (RRRN) for micro-expression recognition that effectively handles occlusion by modeling facial region relations and using attention mechanisms, outperforming existing methods.

Contribution

The paper proposes RRRN, a new network architecture that models relations between facial regions with attention and graph convolutions to improve occlusion-robust micro-expression recognition.

Findings

RRRN outperforms state-of-the-art methods on occlusion datasets.

The attention mechanism effectively suppresses occlusion influence.

Graph-based relation modeling captures complementary facial region interactions.

Abstract

Micro-expression recognition (\textbf{MER}) has attracted lots of researchers' attention in a decade. However, occlusion will occur for MER in real-world scenarios. This paper deeply investigates an interesting but unexplored challenging issue in MER, \ie, occlusion MER. First, to research MER under real-world occlusion, synthetic occluded micro-expression databases are created by using various mask for the community. Second, to suppress the influence of occlusion, a \underline{R}egion-inspired \underline{R}elation \underline{R}easoning \underline{N}etwork (\textbf{RRRN}) is proposed to model relations between various facial regions. RRRN consists of a backbone network, the Region-Inspired (\textbf{RI}) module and Relation Reasoning (\textbf{RR}) module. More specifically, the backbone network aims at extracting feature representations from different facial regions, RI module computing an adaptive weight from the region itself based on attention mechanism with respect to the unobstructedness and importance for suppressing the influence of occlusion, and RR module exploiting the progressive interactions among these regions by performing graph convolutions. Experiments are conducted on handout-database evaluation and composite database evaluation tasks of MEGC 2018 protocol. Experimental results show that RRRN can significantly explore the importance of facial regions and capture the cooperative complementary relationship of facial regions for MER. The results also demonstrate RRRN outperforms the state-of-the-art approaches, especially on occlusion, and RRRN acts more robust to occlusion.