CoCoGaussian: Leveraging Circle of Confusion for Gaussian Splatting from Defocused Images

TL;DR

CoCoGaussian introduces a novel method for 3D scene reconstruction from defocused images by modeling the Circle of Confusion, enabling high-quality rendering even with blurry inputs, and demonstrates state-of-the-art results.

Contribution

It is the first approach to incorporate Circle of Confusion modeling into Gaussian Splatting for defocused images, improving 3D scene reconstruction accuracy.

Findings

Achieves state-of-the-art performance on synthetic datasets.

Effectively handles scenes with reflective and refractive surfaces.

Demonstrates robustness to unreliable depth information.

Abstract

3D Gaussian Splatting (3DGS) has attracted significant attention for its high-quality novel view rendering, inspiring research to address real-world challenges. While conventional methods depend on sharp images for accurate scene reconstruction, real-world scenarios are often affected by defocus blur due to finite depth of field, making it essential to account for realistic 3D scene representation. In this study, we propose CoCoGaussian, a Circle of Confusion-aware Gaussian Splatting that enables precise 3D scene representation using only defocused images. CoCoGaussian addresses the challenge of defocus blur by modeling the Circle of Confusion (CoC) through a physically grounded approach based on the principles of photographic defocus. Exploiting 3D Gaussians, we compute the CoC diameter from depth and learnable aperture information, generating multiple Gaussians to precisely capture the CoC shape. Furthermore, we introduce a learnable scaling factor to enhance robustness and provide more flexibility in handling unreliable depth in scenes with reflective or refractive surfaces. Experiments on both synthetic and real-world datasets demonstrate that CoCoGaussian achieves state-of-the-art performance across multiple benchmarks.