FIELDS: Face reconstruction with accurate Inference of Expression using Learning with Direct Supervision

TL;DR

FIELDS is a novel 3D face reconstruction method that accurately captures subtle emotional expressions by combining 2D consistency with direct 3D supervision and emotion recognition, leading to more realistic and emotionally expressive face models.

Contribution

It introduces a dual-supervision approach with direct 3D expression supervision and an emotion recognition branch, improving emotional detail accuracy in 3D face reconstructions from single images.

Findings

Enhanced emotional expression fidelity in 3D face models.

Improved in-the-wild facial expression recognition performance.

Bridged the 2D/3D domain gap effectively.

Abstract

Facial expressions convey the bulk of emotional information in human communication, yet existing 3D face reconstruction methods often miss subtle affective details due to reliance on 2D supervision and lack of 3D ground truth. We propose FIELDS (Face reconstruction with accurate Inference of Expression using Learning with Direct Supervision) to address these limitations by extending self-supervised 2D image consistency cues with direct 3D expression parameter supervision and an auxiliary emotion recognition branch. Our encoder is guided by authentic expression parameters from spontaneous 4D facial scans, while an intensity-aware emotion loss encourages the 3D expression parameters to capture genuine emotion content without exaggeration. This dual-supervision strategy bridges the 2D/3D domain gap and mitigates expression-intensity bias, yielding high-fidelity 3D reconstructions that preserve subtle emotional cues. From a single image, FIELDS produces emotion-rich face models with highly realistic expressions, significantly improving in-the-wild facial expression recognition performance without sacrificing naturalness.