Single Image Reflection Removal with Physically-Based Training Images

TL;DR

This paper introduces a physically-based rendering approach for synthesizing training data in single image reflection removal, along with a novel network structure and a backtracking module to improve separation quality.

Contribution

It proposes a physically-based data synthesis method and a backtrack network for enhanced reflection separation in single images, advancing beyond prior synthetic data approaches.

Findings

Physically simulated training data improves reflection removal quality.

The backtrack network effectively reduces ghosting and blurring artifacts.

The method outperforms state-of-the-art techniques both visually and numerically.

Abstract

Recently, deep learning-based single image reflection separation methods have been exploited widely. To benefit the learning approach, a large number of training image pairs (i.e., with and without reflections) were synthesized in various ways, yet they are away from a physically-based direction. In this paper, physically based rendering is used for faithfully synthesizing the required training images, and a corresponding network structure and loss term are proposed. We utilize existing RGBD/RGB images to estimate meshes, then physically simulate the light transportation between meshes, glass, and lens with path tracing to synthesize training data, which successfully reproduce the spatially variant anisotropic visual effect of glass reflection. For guiding the separation better, we additionally consider a module, backtrack network (BT-net) for backtracking the reflections, which removes complicated ghosting, attenuation, blurred and defocused effect of glass/lens. This enables obtaining a priori information before having the distortion. The proposed method considering additional a priori information with physically simulated training data is validated with various real reflection images and shows visually pleasant and numerical advantages compared with state-of-the-art techniques.