EXIM: A Hybrid Explicit-Implicit Representation for Text-Guided 3D Shape Generation

TL;DR

This paper introduces EXIM, a hybrid explicit-implicit 3D shape representation technique guided by text, enabling high-fidelity, style-consistent shape generation from natural language without extensive optimization or human annotations.

Contribution

The paper proposes a novel hybrid explicit-implicit shape representation for text-guided 3D shape generation, improving fidelity and coherence over existing methods.

Findings

Outperforms state-of-the-art methods in shape-text coherence

Generates high-quality 3D shapes from natural language descriptions

Enables style-consistent indoor scene generation

Abstract

This paper presents a new text-guided technique for generating 3D shapes. The technique leverages a hybrid 3D shape representation, namely EXIM, combining the strengths of explicit and implicit representations. Specifically, the explicit stage controls the topology of the generated 3D shapes and enables local modifications, whereas the implicit stage refines the shape and paints it with plausible colors. Also, the hybrid approach separates the shape and color and generates color conditioned on shape to ensure shape-color consistency. Unlike the existing state-of-the-art methods, we achieve high-fidelity shape generation from natural-language descriptions without the need for time-consuming per-shape optimization or reliance on human-annotated texts during training or test-time optimization. Further, we demonstrate the applicability of our approach to generate indoor scenes with consistent styles using text-induced 3D shapes. Through extensive experiments, we demonstrate the compelling quality of our results and the high coherency of our generated shapes with the input texts, surpassing the performance of existing methods by a significant margin. Codes and models are released at https://github.com/liuzhengzhe/EXIM.