FLAME-based Multi-View 3D Face Reconstruction

TL;DR

This paper introduces MFNet, a multi-view Flame-based framework for 3D face reconstruction that leverages self-supervised learning and innovative loss functions to improve reconstruction quality in practical scenarios.

Contribution

It presents a novel multi-view training and testing framework combining Flame parametric model with self-supervised learning and new loss functions for enhanced 3D face reconstruction.

Findings

Achieved high-quality 3D face reconstructions on AFLW and Facescape datasets.

Demonstrated effectiveness of multi-view optical flow and landmark constraints.

Validated the model's performance with real-world face images.

Abstract

At present, face 3D reconstruction has broad application prospects in various fields, but the research on it is still in the development stage. In this paper, we hope to achieve better face 3D reconstruction quality by combining multi-view training framework with face parametric model Flame, propose a multi-view training and testing model MFNet (Multi-view Flame Network). We build a self-supervised training framework and implement constraints such as multi-view optical flow loss function and face landmark loss, and finally obtain a complete MFNet. We propose innovative implementations of multi-view optical flow loss and the covisible mask. We test our model on AFLW and facescape datasets and also take pictures of our faces to reconstruct 3D faces while simulating actual scenarios as much as possible, which achieves good results. Our work mainly addresses the problem of combining parametric models of faces with multi-view face 3D reconstruction and explores the implementation of a Flame based multi-view training and testing framework for contributing to the field of face 3D reconstruction.