Fine-Grained 3D Facial Reconstruction for Micro-Expressions

TL;DR

This paper introduces a novel 3D micro-expression reconstruction method that combines global and local features, leveraging macro-expression data to improve accuracy in capturing subtle facial movements.

Contribution

The paper proposes a new fine-grained micro-expression reconstruction approach integrating a dynamic-encoded module and a mesh deformation module, addressing data scarcity and enhancing detail.

Findings

Outperforms state-of-the-art methods in geometric accuracy

Effectively captures subtle micro-expressions with high perceptual detail

Leverages macro-expression data to mitigate micro-expression data scarcity

Abstract

Recent advances in 3D facial expression reconstruction have demonstrated remarkable performance in capturing macro-expressions, yet the reconstruction of micro-expressions remains unexplored. This novel task is particularly challenging due to the subtle, transient, and low-intensity nature of micro-expressions, which complicate the extraction of stable and discriminative features essential for accurate reconstruction. In this paper, we propose a fine-grained micro-expression reconstruction method that integrates a global dynamic feature capturing stable facial motion patterns with a locally-enriched feature incorporating multiple informative cues from 2D motions, facial priors and 3D facial geometry. Specifically, we devise a plug-and-play dynamic-encoded module to extract micro-expression feature for global facial action, allowing it to leverage prior knowledge from abundant macro-expression data to mitigate the scarcity of micro-expression data. Subsequently, a dynamic-guided mesh deformation module is designed for extracting aggregated local features from dense optical flow, sparse landmark cues and facial mesh geometry, which adaptively refines fine-grained facial micro-expression without compromising global 3D geometry. Extensive experiments on micro-expression datasets demonstrate that our method consistently outperforms state-of-the-art methods in both geometric accuracy and perceptual detail.