LoomNet: Enhancing Multi-View Image Generation via Latent Space Weaving

TL;DR

LoomNet introduces a multi-view diffusion architecture that collaboratively builds a shared latent space to generate coherent, high-quality multi-view images efficiently, improving consistency and diversity over existing methods.

Contribution

This paper presents LoomNet, a novel diffusion-based model that enhances multi-view image generation by leveraging shared latent space weaving for view consistency.

Findings

Outperforms state-of-the-art in image quality and reconstruction metrics

Generates 16 high-quality views in 15 seconds

Produces diverse, plausible novel views from a single input

Abstract

Generating consistent multi-view images from a single image remains challenging. Lack of spatial consistency often degrades 3D mesh quality in surface reconstruction. To address this, we propose LoomNet, a novel multi-view diffusion architecture that produces coherent images by applying the same diffusion model multiple times in parallel to collaboratively build and leverage a shared latent space for view consistency. Each viewpoint-specific inference generates an encoding representing its own hypothesis of the novel view from a given camera pose, which is projected onto three orthogonal planes. For each plane, encodings from all views are fused into a single aggregated plane. These aggregated planes are then processed to propagate information and interpolate missing regions, combining the hypotheses into a unified, coherent interpretation. The final latent space is then used to render consistent multi-view images. LoomNet generates 16 high-quality and coherent views in just 15 seconds. In our experiments, LoomNet outperforms state-of-the-art methods on both image quality and reconstruction metrics, also showing creativity by producing diverse, plausible novel views from the same input.