DOF-GS:Adjustable Depth-of-Field 3D Gaussian Splatting for Post-Capture Refocusing, Defocus Rendering and Blur Removal

TL;DR

DOF-GS introduces a 3D Gaussian Splatting framework with a finite-aperture model that enables post-capture refocusing, defocus control, and high-quality all-in-focus rendering from multi-view images.

Contribution

It presents a novel 3DGS-based method with explicit defocus modeling, allowing post-capture focus adjustment and defocus rendering, which was not possible with previous pinhole camera models.

Findings

Supports post-capture refocusing and defocus adjustment.

Learns camera characteristics from images with moderate defocus.

Produces high-quality all-in-focus renderings from uncalibrated multi-view images.

Abstract

3D Gaussian Splatting (3DGS) techniques have recently enabled high-quality 3D scene reconstruction and real-time novel view synthesis. These approaches, however, are limited by the pinhole camera model and lack effective modeling of defocus effects. Departing from this, we introduce DOF-GS--a new 3DGS-based framework with a finite-aperture camera model and explicit, differentiable defocus rendering, enabling it to function as a post-capture control tool. By training with multi-view images with moderate defocus blur, DOF-GS learns inherent camera characteristics and reconstructs sharp details of the underlying scene, particularly, enabling rendering of varying DOF effects through on-demand aperture and focal distance control, post-capture and optimization. Additionally, our framework extracts circle-of-confusion cues during optimization to identify in-focus regions in input views, enhancing the reconstructed 3D scene details. Experimental results demonstrate that DOF-GS supports post-capture refocusing, adjustable defocus and high-quality all-in-focus rendering, from multi-view images with uncalibrated defocus blur.