Local Geometric Indexing of High Resolution Data for Facial Reconstruction from Sparse Markers

TL;DR

This paper introduces a method for facial reconstruction from sparse markers by indexing high-resolution data locally, leveraging extensive datasets and physical simulations to improve accuracy over traditional blendshape models.

Contribution

It proposes a novel local geometric indexing approach that uses high-resolution datasets and physical simulations to enhance facial reconstruction from sparse markers.

Findings

Effective reconstruction accuracy demonstrated

Enhanced dataset diversity improves results

Physical simulation augmentation benefits the method

Abstract

When considering sparse motion capture marker data, one typically struggles to balance its overfitting via a high dimensional blendshape system versus underfitting caused by smoothness constraints. With the current trend towards using more and more data, our aim is not to fit the motion capture markers with a parameterized (blendshape) model or to smoothly interpolate a surface through the marker positions, but rather to find an instance in the high resolution dataset that contains local geometry to fit each marker. Just as is true for typical machine learning applications, this approach benefits from a plethora of data, and thus we also consider augmenting the dataset via specially designed physical simulations that target the high resolution dataset such that the simulation output lies on the same so-called manifold as the data targeted.