Sel3DCraft: Interactive Visual Prompts for User-Friendly Text-to-3D Generation

TL;DR

Sel3DCraft is an interactive system that improves text-to-3D generation by guiding users through visual prompts, multi-view evaluation, and defect refinement, enhancing creativity and usability.

Contribution

It introduces a novel visual prompt engineering framework with a dual-branch exploration, multi-view scoring, and an analytics suite for better 3D model generation.

Findings

Outperforms existing T23D systems in supporting designer creativity

Enhances multi-view consistency assessment with high-level metrics

Enables intuitive defect identification and refinement

Abstract

Text-to-3D (T23D) generation has transformed digital content creation, yet remains bottlenecked by blind trial-and-error prompting processes that yield unpredictable results. While visual prompt engineering has advanced in text-to-image domains, its application to 3D generation presents unique challenges requiring multi-view consistency evaluation and spatial understanding. We present Sel3DCraft, a visual prompt engineering system for T23D that transforms unstructured exploration into a guided visual process. Our approach introduces three key innovations: a dual-branch structure combining retrieval and generation for diverse candidate exploration; a multi-view hybrid scoring approach that leverages MLLMs with innovative high-level metrics to assess 3D models with human-expert consistency; and a prompt-driven visual analytics suite that enables intuitive defect identification and refinement. Extensive testing and user studies demonstrate that Sel3DCraft surpasses other T23D systems in supporting creativity for designers.