KaoLRM: Repurposing Pre-trained Large Reconstruction Models for Parametric 3D Face Reconstruction

TL;DR

KaoLRM leverages pre-trained large reconstruction models and integrates FLAME-based 2D Gaussian Splatting to improve the accuracy and consistency of 3D face reconstructions from single images, especially across different viewpoints.

Contribution

This work introduces KaoLRM, a novel method that re-purposes pre-trained large reconstruction models for robust parametric 3D face reconstruction using FLAME-based features.

Findings

Achieves superior accuracy on controlled and in-the-wild benchmarks.

Demonstrates improved cross-view consistency over existing methods.

Robustly handles self-occlusions and diverse viewpoints.

Abstract

We propose KaoLRM to re-target the learned prior of the Large Reconstruction Model (LRM) for parametric 3D face reconstruction from single-view images. Parametric 3D Morphable Models (3DMMs) have been widely used for facial reconstruction due to their compact and interpretable parameterization, yet existing 3DMM regressors often exhibit poor consistency across varying viewpoints. To address this, we harness the pre-trained 3D prior of LRM and incorporate FLAME-based 2D Gaussian Splatting into LRM's rendering pipeline. Specifically, KaoLRM projects LRM's pre-trained triplane features into the FLAME parameter space to recover geometry, and models appearance via 2D Gaussian primitives that are tightly coupled to the FLAME mesh. The rich prior enables the FLAME regressor to be aware of the 3D structure, leading to accurate and robust reconstructions under self-occlusions and diverse viewpoints. Experiments on both controlled and in-the-wild benchmarks demonstrate that KaoLRM achieves superior reconstruction accuracy and cross-view consistency, while existing methods remain sensitive to viewpoint variations. The code is released at https://github.com/CyberAgentAILab/KaoLRM.