3D Human Face Reconstruction with 3DMM face model from RGB image

TL;DR

This paper introduces a pipeline that reconstructs detailed 3D human face models from single RGB images using CNNs, landmark detection, 3DMM regression, and soft rendering.

Contribution

It presents a novel end-to-end pipeline combining face detection, landmark detection, 3DMM parameter regression, and soft rendering for detailed 3D face reconstruction from RGB images.

Findings

Achieves detailed 3D face reconstruction from single RGB images.

Utilizes a pipeline integrating CNN-based landmark detection and 3DMM regression.

Provides code repositories for implementation and further research.

Abstract

Nowadays as convolution neural networks demonstrate its powerful problem-solving ability in the area of image processing, efforts have been made to reconstruct detailed face shapes from 2D face images or videos. However, to make the full use of CNN, a large number of labeled data is required to train the network. Coarse morphable face model has been used to synthesize labeled data. However, it is hard for coarse morphable face models to generate photo-realistic data with detail such as wrinkles. In this project, we present a pipeline that reconstructs a human face 3D model from a single RGB image. The pipeline includes face detection, landmark detection, regression of 3DMM model parameters, and soft rendering. Mentor: Zhipeng Fan (Email: zf606@nyu.edu) Code Repository: https://github.com/SeVEnMY/3d-face- reconstruction Code Reference: https://github.com/sicxu/Deep3DFaceRecon pytorch