More comprehensive facial inversion for more effective expression recognition

TL;DR

This paper introduces a novel generative approach using image inversion and a transformer-based model to improve facial expression recognition by extracting richer facial features.

Contribution

It proposes a new inversion-based generative method with a transformer architecture and feature modulation for enhanced FER performance.

Findings

Achieves state-of-the-art facial inversion results on FFHQ and CelebA-HQ.

Attains competitive FER results on RAF-DB, SFEW, and AffectNet.

Demonstrates the effectiveness of the proposed ASIT in feature extraction.

Abstract

Facial expression recognition (FER) plays a significant role in the ubiquitous application of computer vision. We revisit this problem with a new perspective on whether it can acquire useful representations that improve FER performance in the image generation process, and propose a novel generative method based on the image inversion mechanism for the FER task, termed Inversion FER (IFER). Particularly, we devise a novel Adversarial Style Inversion Transformer (ASIT) towards IFER to comprehensively extract features of generated facial images. In addition, ASIT is equipped with an image inversion discriminator that measures the cosine similarity of semantic features between source and generated images, constrained by a distribution alignment loss. Finally, we introduce a feature modulation module to fuse the structural code and latent codes from ASIT for the subsequent FER work. We extensively evaluate ASIT on facial datasets such as FFHQ and CelebA-HQ, showing that our approach achieves state-of-the-art facial inversion performance. IFER also achieves competitive results in facial expression recognition datasets such as RAF-DB, SFEW and AffectNet. The code and models are available at https://github.com/Talented-Q/IFER-master.