Revisiting an Old Perspective Projection for Monocular 3D Morphable Models Regression

TL;DR

This paper proposes a new camera model for monocular 3D Morphable Model regression that captures perspective distortion in close-up facial images, improving accuracy for head-mounted camera footage.

Contribution

It introduces a pseudo-perspective extension to orthographic projection, enabling better modeling of close-up images while maintaining stability and compatibility with existing models.

Findings

Enhanced modeling of perspective effects in close-up images

Improved accuracy in 3D face reconstruction from head-mounted camera footage

Effective finetuning techniques demonstrated on a custom dataset

Abstract

We introduce a novel camera model for monocular 3D Morphable Model (3DMM) regression methods that effectively captures the perspective distortion effect commonly seen in close-up facial images. Fitting 3D morphable models to video is a key technique in content creation. In particular, regression-based approaches have produced fast and accurate results by matching the rendered output of the morphable model to the target image. These methods typically achieve stable performance with orthographic projection, which eliminates the ambiguity between focal length and object distance. However, this simplification makes them unsuitable for close-up footage, such as that captured with head-mounted cameras. We extend orthographic projection with a new shrinkage parameter, incorporating a pseudo-perspective effect while preserving the stability of the original projection. We present several techniques that allow finetuning of existing models, and demonstrate the effectiveness of our modification through both quantitative and qualitative comparisons using a custom dataset recorded with head-mounted cameras.