Parameter-Free Neural Lens Blur Rendering for High-Fidelity Composites

TL;DR

This paper introduces a parameter-free neural method for rendering realistic lens blur in compositing virtual objects into real scenes, eliminating the need for scene depth or camera parameters.

Contribution

It proposes a novel approach that estimates the circle of confusion directly from RGB images and uses neural reblurring for high-fidelity lens blur rendering.

Findings

Achieves high-fidelity, realistic lens blur in composites

Outperforms state-of-the-art methods in qualitative evaluations

Operates without scene depth or camera metadata

Abstract

Consistent and natural camera lens blur is important for seamlessly blending 3D virtual objects into photographed real-scenes. Since lens blur typically varies with scene depth, the placement of virtual objects and their corresponding blur levels significantly affect the visual fidelity of mixed reality compositions. Existing pipelines often rely on camera parameters (e.g., focal length, focus distance, aperture size) and scene depth to compute the circle of confusion (CoC) for realistic lens blur rendering. However, such information is often unavailable to ordinary users, limiting the accessibility and generalizability of these methods. In this work, we propose a novel compositing approach that directly estimates the CoC map from RGB images, bypassing the need for scene depth or camera metadata. The CoC values for virtual objects are inferred through a linear relationship between its signed CoC map and depth, and realistic lens blur is rendered using a neural reblurring network. Our method provides flexible and practical solution for real-world applications. Experimental results demonstrate that our method achieves high-fidelity compositing with realistic defocus effects, outperforming state-of-the-art techniques in both qualitative and quantitative evaluations.