Twist and Compute: The Cost of Pose in 3D Generative Diffusion

TL;DR

This paper investigates the limitations of large-scale image-to-3D generative models, revealing a strong canonical view bias and proposing a lightweight correction method to improve viewpoint generalization.

Contribution

The study identifies a canonical view bias in 3D generative models and introduces a simple CNN-based correction approach to mitigate it without altering the core model.

Findings

Model performance degrades with rotated inputs.

A lightweight CNN can correct input orientation effectively.

The correction restores model performance across viewpoints.

Abstract

Despite their impressive results, large-scale image-to-3D generative models remain opaque in their inductive biases. We identify a significant limitation in image-conditioned 3D generative models: a strong canonical view bias. Through controlled experiments using simple 2D rotations, we show that the state-of-the-art Hunyuan3D 2.0 model can struggle to generalize across viewpoints, with performance degrading under rotated inputs. We show that this failure can be mitigated by a lightweight CNN that detects and corrects input orientation, restoring model performance without modifying the generative backbone. Our findings raise an important open question: Is scale enough, or should we pursue modular, symmetry-aware designs?