Text-Image Conditioned 3D Generation

TL;DR

This paper introduces a novel approach for 3D content generation that combines text and image inputs to leverage their complementary strengths, resulting in more flexible and detailed 3D asset creation.

Contribution

The paper formalizes the task of text-image conditioned 3D generation and proposes TIGON, a dual-branch model that effectively fuses visual and textual information for improved 3D synthesis.

Findings

Cross-modal fusion outperforms single-modality models.

Text-image conditioning enhances 3D generation quality.

TIGON demonstrates consistent improvements across experiments.

Abstract

High-quality 3D assets are essential for VR/AR, industrial design, and entertainment, motivating growing interest in generative models that create 3D content from user prompts. Most existing 3D generators, however, rely on a single conditioning modality: image-conditioned models achieve high visual fidelity by exploiting pixel-aligned cues but suffer from viewpoint bias when the input view is limited or ambiguous, while text-conditioned models provide broad semantic guidance yet lack low-level visual detail. This limits how users can express intent and raises a natural question: can these two modalities be combined for more flexible and faithful 3D generation? Our diagnostic study shows that even simple late fusion of text- and image-conditioned predictions outperforms single-modality models, revealing strong cross-modal complementarity. We therefore formalize Text-Image Conditioned 3D Generation, which requires joint reasoning over a visual exemplar and a textual specification. To address this task, we introduce TIGON, a minimalist dual-branch baseline with separate image- and text-conditioned backbones and lightweight cross-modal fusion. Extensive experiments show that text-image conditioning consistently improves over single-modality methods, highlighting complementary vision-language guidance as a promising direction for future 3D generation research. Project page: https://jumpat.github.io/tigon-page