Freeplane: Unlocking Free Lunch in Triplane-Based Sparse-View Reconstruction Models

TL;DR

Freeplane is a simple, cost-free method that enhances 3D reconstruction quality from single-view images by filtering and combining triplane features during inference, without additional training.

Contribution

It introduces a novel frequency modulation technique for triplanes that improves multi-view consistency and 3D mesh quality in feed-forward models without retraining.

Findings

Significant improvement in 3D mesh quality and texture fidelity.

Effective reduction of high-frequency artifacts in generated triplanes.

Seamless integration with existing pre-trained models during inference.

Abstract

Creating 3D assets from single-view images is a complex task that demands a deep understanding of the world. Recently, feed-forward 3D generative models have made significant progress by training large reconstruction models on extensive 3D datasets, with triplanes being the preferred 3D geometry representation. However, effectively utilizing the geometric priors of triplanes, while minimizing artifacts caused by generated inconsistent multi-view images, remains a challenge. In this work, we present \textbf{Fre}quency modulat\textbf{e}d tri\textbf{plane} (\textbf{Freeplane}), a simple yet effective method to improve the generation quality of feed-forward models without additional training. We first analyze the role of triplanes in feed-forward methods and find that the inconsistent multi-view images introduce high-frequency artifacts on triplanes, leading to low-quality 3D meshes. Based on this observation, we propose strategically filtering triplane features and combining triplanes before and after filtering to produce high-quality textured meshes. These techniques incur no additional cost and can be seamlessly integrated into pre-trained feed-forward models to enhance their robustness against the inconsistency of generated multi-view images. Both qualitative and quantitative results demonstrate that our method improves the performance of feed-forward models by simply modulating triplanes. All you need is to modulate the triplanes during inference.