Generative AI meets 3D: A Survey on Text-to-3D in AIGC Era

TL;DR

This survey comprehensively reviews recent advances in text-to-3D generative AI, covering data representations, core technologies, benchmarks, applications, and future research directions in the emerging field.

Contribution

It provides a systematic overview of the state-of-the-art in text-to-3D, highlighting recent progress, core methods, and future challenges in the field.

Findings

Summarizes key data representations for 3D models.

Reviews core technologies like NeRF for text-to-3D generation.

Outlines main applications including avatar and scene generation.

Abstract

Generative AI has made significant progress in recent years, with text-guided content generation being the most practical as it facilitates interaction between human instructions and AI-generated content (AIGC). Thanks to advancements in text-to-image and 3D modeling technologies, like neural radiance field (NeRF), text-to-3D has emerged as a nascent yet highly active research field. Our work conducts a comprehensive survey on this topic and follows up on subsequent research progress in the overall field, aiming to help readers interested in this direction quickly catch up with its rapid development. First, we introduce 3D data representations, including both Structured and non-Structured data. Building on this pre-requisite, we introduce various core technologies to achieve satisfactory text-to-3D results. Additionally, we present mainstream baselines and research directions in recent text-to-3D technology, including fidelity, efficiency, consistency, controllability, diversity, and applicability. Furthermore, we summarize the usage of text-to-3D technology in various applications, including avatar generation, texture generation, scene generation and 3D editing. Finally, we discuss the agenda for the future development of text-to-3D.