FC-4DFS: Frequency-controlled Flexible 4D Facial Expression Synthesizing

TL;DR

This paper introduces FC-4DFS, a novel frequency-controlled LSTM-based method for generating smooth, flexible 4D facial expression sequences with improved temporal coherence, outperforming existing approaches on standard datasets.

Contribution

The paper presents a frequency-controlled LSTM network and a multi-level identity-aware displacement network for more flexible and accurate 4D facial expression synthesis.

Findings

Achieves state-of-the-art results on CoMA and Florence4D datasets.

Enhances temporal coherence in generated sequences.

Provides flexible sequence length generation.

Abstract

4D facial expression synthesizing is a critical problem in the fields of computer vision and graphics. Current methods lack flexibility and smoothness when simulating the inter-frame motion of expression sequences. In this paper, we propose a frequency-controlled 4D facial expression synthesizing method, FC-4DFS. Specifically, we introduce a frequency-controlled LSTM network to generate 4D facial expression sequences frame by frame from a given neutral landmark with a given length. Meanwhile, we propose a temporal coherence loss to enhance the perception of temporal sequence motion and improve the accuracy of relative displacements. Furthermore, we designed a Multi-level Identity-Aware Displacement Network based on a cross-attention mechanism to reconstruct the 4D facial expression sequences from landmark sequences. Finally, our FC-4DFS achieves flexible and SOTA generation results of 4D facial expression sequences with different lengths on CoMA and Florence4D datasets. The code will be available on GitHub.