LoGoColor: Local-Global 3D Colorization for 360{\deg} Scenes

TL;DR

LoGoColor introduces a novel local-global approach for 3D colorization of 360-degree scenes, enhancing color diversity and multi-view consistency over existing methods that tend to oversimplify colors.

Contribution

The paper presents a new pipeline that preserves color diversity and ensures multi-view consistency by partitioning scenes and applying a multi-view diffusion model.

Findings

Achieves more consistent and plausible 3D colorization results.

Outperforms existing methods in both quantitative and qualitative evaluations.

Effectively handles complex 360-degree scenes with enhanced color diversity.

Abstract

Single-channel 3D reconstruction is widely used in fields such as robotics and medical imaging. While these methods are good at reconstructing 3D geometry, their outputs are typically uncolored 3D models, making 3D colorization necessary for visualization. Recent 3D colorization studies address this problem by distilling 2D image colorization models. However, these approaches suffer from an inherent inconsistency of 2D image models. This results in colors being averaged during training, leading to monotonous and oversimplified results, particularly in complex 360{\deg} scenes. In contrast, we aim to preserve color diversity by generating a new set of consistently colorized training views, thereby suppressing the averaging process. Nevertheless, mitigating the averaging process introduces a new challenge: ensuring strict multi-view consistency across these colorized views. To achieve this, we propose \ourmethod, a pipeline designed to preserve color diversity by eliminating this guidance-averaging process with a `Local-Global' approach: we partition the scene into subscenes and explicitly tackle both inter-subscene and intra-subscene consistency using a fine-tuned multi-view diffusion model. We demonstrate our method achieves quantitatively and qualitatively more consistent and plausible 3D colorization on complex 360{\deg} scenes than existing methods.