GeoMVD: Geometry-Enhanced Multi-View Generation Model Based on Geometric Information Extraction

TL;DR

GeoMVD introduces a geometry-guided multi-view diffusion model that extracts and leverages geometric information to generate consistent, high-quality multi-view images for applications like 3D reconstruction and virtual reality.

Contribution

The paper presents a novel geometry extraction module, a decoupled attention mechanism, and an adaptive intensity adjustment strategy to improve multi-view image generation quality and consistency.

Findings

Enhanced cross-view consistency in generated images.

Improved detail preservation and image quality.

Effective geometric information utilization in multi-view synthesis.

Abstract

Multi-view image generation holds significant application value in computer vision, particularly in domains like 3D reconstruction, virtual reality, and augmented reality. Most existing methods, which rely on extending single images, face notable computational challenges in maintaining cross-view consistency and generating high-resolution outputs. To address these issues, we propose the Geometry-guided Multi-View Diffusion Model, which incorporates mechanisms for extracting multi-view geometric information and adjusting the intensity of geometric features to generate images that are both consistent across views and rich in detail. Specifically, we design a multi-view geometry information extraction module that leverages depth maps, normal maps, and foreground segmentation masks to construct a shared geometric structure, ensuring shape and structural consistency across different views. To enhance consistency and detail restoration during generation, we develop a decoupled geometry-enhanced attention mechanism that strengthens feature focus on key geometric details, thereby improving overall image quality and detail preservation. Furthermore, we apply an adaptive learning strategy that fine-tunes the model to better capture spatial relationships and visual coherence between the generated views, ensuring realistic results. Our model also incorporates an iterative refinement process that progressively improves the output quality through multiple stages of image generation. Finally, a dynamic geometry information intensity adjustment mechanism is proposed to adaptively regulate the influence of geometric data, optimizing overall quality while ensuring the naturalness of generated images. More details can be found on the project page: https://sobeymil.github.io/GeoMVD.com.