BridgeShape: Latent Diffusion Schr\"odinger Bridge for 3D Shape Completion

TL;DR

BridgeShape introduces a novel latent diffusion Schr"odinger bridge framework for 3D shape completion, explicitly modeling optimal transport and utilizing depth-enhanced VQ-VAE for high-fidelity results.

Contribution

It formulates shape completion as an optimal transport problem and employs a depth-enhanced VQ-VAE to operate in a compact, structurally rich latent space.

Findings

Achieves state-of-the-art performance on large-scale benchmarks.

Produces higher resolution and more detailed 3D shape completions.

Effective across diverse unseen object classes.

Abstract

Existing diffusion-based 3D shape completion methods typically use a conditional paradigm, injecting incomplete shape information into the denoising network via deep feature interactions (e.g., concatenation, cross-attention) to guide sampling toward complete shapes, often represented by voxel-based distance functions. However, these approaches fail to explicitly model the optimal global transport path, leading to suboptimal completions. Moreover, performing diffusion directly in voxel space imposes resolution constraints, limiting the generation of fine-grained geometric details. To address these challenges, we propose BridgeShape, a novel framework for 3D shape completion via latent diffusion Schr\"odinger bridge. The key innovations lie in two aspects: (i) BridgeShape formulates shape completion as an optimal transport problem, explicitly modeling the transition between incomplete and complete shapes to ensure a globally coherent transformation. (ii) We introduce a Depth-Enhanced Vector Quantized Variational Autoencoder (VQ-VAE) to encode 3D shapes into a compact latent space, leveraging self-projected multi-view depth information enriched with strong DINOv2 features to enhance geometric structural perception. By operating in a compact yet structurally informative latent space, BridgeShape effectively mitigates resolution constraints and enables more efficient and high-fidelity 3D shape completion. BridgeShape achieves state-of-the-art performance on large-scale 3D shape completion benchmarks, demonstrating superior fidelity at higher resolutions and for unseen object classes.