Real-Time Monocular 4D Face Reconstruction using the LSFM models

TL;DR

This paper presents a real-time system for monocular 4D face reconstruction using LSFM models, capable of capturing and visualizing facial identity and expressions from a single webcam on a standard laptop.

Contribution

It introduces the first real-time demo leveraging LSFM models for accurate 4D face reconstruction from monocular input.

Findings

Achieves real-time performance on a standard laptop

Accurately reconstructs facial identity and expressions

Demonstrates practical usability of LSFM models in real-time applications

Abstract

4D face reconstruction from a single camera is a challenging task, especially when it is required to be performed in real time. We demonstrate a system of our own implementation that solves this task accurately and runs in real time on a commodity laptop, using a webcam as the only input. Our system is interactive, allowing the user to freely move their head and show various expressions while standing in front of the camera. As a result, the put forward system both reconstructs and visualises the identity of the subject in the correct pose along with the acted facial expressions in real-time. The 4D reconstruction in our framework is based on the recently-released Large-Scale Facial Models (LSFM) \cite{LSFM1, LSFM2}, which are the largest-scale 3D Morphable Models of facial shapes ever constructed, based on a dataset of more than 10,000 facial identities from a wide range of gender, age and ethnicity combinations. This is the first real-time demo that gives users the opportunity to test in practice the capabilities of the recently-released Large-Scale Facial Models (LSFM)