Cue3D: Quantifying the Role of Image Cues in Single-Image 3D Generation

TL;DR

Cue3D introduces a comprehensive framework to quantify how different image cues like shading, texture, and silhouette influence single-image 3D generation, revealing the importance of shape and geometric cues for model performance.

Contribution

It provides the first model-agnostic benchmark to systematically evaluate the influence of individual image cues on 3D generation methods.

Findings

Shape meaningfulness is key for generalization.

Shading is a crucial geometric cue for 3D generation.

Models over-rely on silhouettes and show varied sensitivities to cues.

Abstract

Humans and traditional computer vision methods rely on a diverse set of monocular cues to infer 3D structure from a single image, such as shading, texture, silhouette, etc. While recent deep generative models have dramatically advanced single-image 3D generation, it remains unclear which image cues these methods actually exploit. We introduce Cue3D, the first comprehensive, model-agnostic framework for quantifying the influence of individual image cues in single-image 3D generation. Our unified benchmark evaluates seven state-of-the-art methods, spanning regression-based, multi-view, and native 3D generative paradigms. By systematically perturbing cues such as shading, texture, silhouette, perspective, edges, and local continuity, we measure their impact on 3D output quality. Our analysis reveals that shape meaningfulness, not texture, dictates generalization. Geometric cues, particularly shading, are crucial for 3D generation. We further identify over-reliance on provided silhouettes and diverse sensitivities to cues such as perspective and local continuity across model families. By dissecting these dependencies, Cue3D advances our understanding of how modern 3D networks leverage classical vision cues, and offers directions for developing more transparent, robust, and controllable single-image 3D generation models.