MPIB: An MPI-Based Bokeh Rendering Framework for Realistic Partial Occlusion Effects

TL;DR

This paper introduces MPIB, a novel MPI-based high-resolution bokeh rendering framework that effectively simulates realistic partial occlusion effects, overcoming limitations of existing methods.

Contribution

The paper proposes a new MPI-based approach combined with background inpainting for high-resolution bokeh rendering, enabling realistic partial occlusion effects.

Findings

Effective simulation of partial occlusion effects.

High-resolution bokeh rendering achieved.

Validated on synthesized and real-world images.

Abstract

Partial occlusion effects are a phenomenon that blurry objects near a camera are semi-transparent, resulting in partial appearance of occluded background. However, it is challenging for existing bokeh rendering methods to simulate realistic partial occlusion effects due to the missing information of the occluded area in an all-in-focus image. Inspired by the learnable 3D scene representation, Multiplane Image (MPI), we attempt to address the partial occlusion by introducing a novel MPI-based high-resolution bokeh rendering framework, termed MPIB. To this end, we first present an analysis on how to apply the MPI representation to bokeh rendering. Based on this analysis, we propose an MPI representation module combined with a background inpainting module to implement high-resolution scene representation. This representation can then be reused to render various bokeh effects according to the controlling parameters. To train and test our model, we also design a ray-tracing-based bokeh generator for data generation. Extensive experiments on synthesized and real-world images validate the effectiveness and flexibility of this framework.