ShapeGen: Towards High-Quality 3D Shape Synthesis

TL;DR

ShapeGen introduces a novel approach for high-quality 3D shape synthesis from images, overcoming previous limitations in detail, surface smoothness, and structural integrity, and setting new state-of-the-art results.

Contribution

The paper presents ShapeGen, a new method that improves 3D shape generation by enhancing representation, supervision, resolution, and transformer use, leading to superior quality.

Findings

Achieves high-fidelity 3D assets with detailed features

Surpasses previous methods in quality and realism

Validates improvements through extensive experiments

Abstract

Inspired by generative paradigms in image and video, 3D shape generation has made notable progress, enabling the rapid synthesis of high-fidelity 3D assets from a single image. However, current methods still face challenges, including the lack of intricate details, overly smoothed surfaces, and fragmented thin-shell structures. These limitations leave the generated 3D assets still one step short of meeting the standards favored by artists. In this paper, we present ShapeGen, which achieves high-quality image-to-3D shape generation through 3D representation and supervision improvements, resolution scaling up, and the advantages of linear transformers. These advancements allow the generated assets to be seamlessly integrated into 3D pipelines, facilitating their widespread adoption across various applications. Through extensive experiments, we validate the impact of these improvements on overall performance. Ultimately, thanks to the synergistic effects of these enhancements, ShapeGen achieves a significant leap in image-to-3D generation, establishing a new state-of-the-art performance.