GeoFusionLRM: Geometry-Aware Self-Correction for Consistent 3D Reconstruction

TL;DR

GeoFusionLRM is a novel framework that enhances single-image 3D reconstruction by self-correcting geometric inconsistencies using the model's own predictions, leading to more accurate and consistent 3D models.

Contribution

It introduces a geometry-aware self-correction mechanism with a transformer-based feedback system, improving reconstruction fidelity without extra supervision.

Findings

Achieves sharper geometry and more consistent normals.

Outperforms state-of-the-art LRM baselines in fidelity.

Enforces geometric consistency with the input image.

Abstract

Single-image 3D reconstruction with large reconstruction models (LRMs) has advanced rapidly, yet reconstructions often exhibit geometric inconsistencies and misaligned details that limit fidelity. We introduce GeoFusionLRM, a geometry-aware self-correction framework that leverages the model's own normal and depth predictions to refine structural accuracy. Unlike prior approaches that rely solely on features extracted from the input image, GeoFusionLRM feeds back geometric cues through a dedicated transformer and fusion module, enabling the model to correct errors and enforce consistency with the conditioning image. This design improves the alignment between the reconstructed mesh and the input views without additional supervision or external signals. Extensive experiments demonstrate that GeoFusionLRM achieves sharper geometry, more consistent normals, and higher fidelity than state-of-the-art LRM baselines.