Single Image to High-Quality 3D Object via Latent Features

TL;DR

LatentDreamer is a new framework that efficiently generates high-fidelity 3D objects from single images using a pre-trained autoencoder to map 3D geometries to latent features, enabling fast and detailed 3D creation.

Contribution

It introduces LatentDreamer, a novel method leveraging a pre-trained variational autoencoder to improve speed and detail in single-image 3D object generation.

Findings

Generates 3D objects in about 70 seconds.

Achieves high fidelity to input images.

Performs competitively with limited training data.

Abstract

3D assets are essential in the digital age. While automatic 3D generation, such as image-to-3d, has made significant strides in recent years, it often struggles to achieve fast, detailed, and high-fidelity generation simultaneously. In this work, we introduce LatentDreamer, a novel framework for generating 3D objects from single images. The key to our approach is a pre-trained variational autoencoder that maps 3D geometries to latent features, which greatly reducing the difficulty of 3D generation. Starting from latent features, the pipeline of LatentDreamer generates coarse geometries, refined geometries, and realistic textures sequentially. The 3D objects generated by LatentDreamer exhibit high fidelity to the input images, and the entire generation process can be completed within a short time (typically in 70 seconds). Extensive experiments show that with only a small amount of training, LatentDreamer demonstrates competitive performance compared to contemporary approachs.