Deep Insights of Learning based Micro Expression Recognition: A Perspective on Promises, Challenges and Research Needs

TL;DR

This paper provides a comprehensive analysis of deep learning approaches for micro expression recognition, highlighting design strategies, experimental protocols, challenges, and future research directions in this challenging field.

Contribution

It offers a detailed categorization of DL-based MER frameworks and empirical analysis of experimental strategies, addressing gaps in understanding their impact.

Findings

Deep learning frameworks vary significantly in design and validation.

Experimental protocols influence the performance and comparability of MER methods.

Identified challenges and future research directions for MER.

Abstract

Micro expression recognition (MER) is a very challenging area of research due to its intrinsic nature and fine-grained changes. In the literature, the problem of MER has been solved through handcrafted/descriptor-based techniques. However, in recent times, deep learning (DL) based techniques have been adopted to gain higher performance for MER. Also, rich survey articles on MER are available by summarizing the datasets, experimental settings, conventional and deep learning methods. In contrast, these studies lack the ability to convey the impact of network design paradigms and experimental setting strategies for DL-based MER. Therefore, this paper aims to provide a deep insight into the DL-based MER frameworks with a perspective on promises in network model designing, experimental strategies, challenges, and research needs. Also, the detailed categorization of available MER frameworks is presented in various aspects of model design and technical characteristics. Moreover, an empirical analysis of the experimental and validation protocols adopted by MER methods is presented. The challenges mentioned earlier and network design strategies may assist the affective computing research community in forging ahead in MER research. Finally, we point out the future directions, research needs, and draw our conclusions.