MPT: Motion Prompt Tuning for Micro-Expression Recognition

TL;DR

This paper proposes Motion Prompt Tuning (MPT), a novel method that adapts large language models for micro-expression recognition by capturing subtle facial motions, significantly improving performance on benchmark datasets.

Contribution

Introduction of Motion Prompt Tuning (MPT), including motion prompt generation and a group adapter, to effectively adapt language models for micro-expression recognition.

Findings

MPT outperforms state-of-the-art methods on three MER datasets.

Motion prompt generation effectively captures subtle facial motions.

The group adapter enhances model sensitivity to micro-expressions.

Abstract

Micro-expression recognition (MER) is crucial in the affective computing field due to its wide application in medical diagnosis, lie detection, and criminal investigation. Despite its significance, obtaining micro-expression (ME) annotations is challenging due to the expertise required from psychological professionals. Consequently, ME datasets often suffer from a scarcity of training samples, severely constraining the learning of MER models. While current large pre-training models (LMs) offer general and discriminative representations, their direct application to MER is hindered by an inability to capture transitory and subtle facial movements-essential elements for effective MER. This paper introduces Motion Prompt Tuning (MPT) as a novel approach to adapting LMs for MER, representing a pioneering method for subtle motion prompt tuning. Particularly, we introduce motion prompt generation, including motion magnification and Gaussian tokenization, to extract subtle motions as prompts for LMs. Additionally, a group adapter is carefully designed and inserted into the LM to enhance it in the target MER domain, facilitating a more nuanced distinction of ME representation. Furthermore, extensive experiments conducted on three widely used MER datasets demonstrate that our proposed MPT consistently surpasses state-of-the-art approaches and verifies its effectiveness.