Separating Reflection and Transmission Images in the Wild

TL;DR

This paper introduces a deep learning method that leverages polarization properties and realistic synthetic data to effectively separate reflection and transmission layers in real-world images, improving upon previous approaches.

Contribution

The paper presents a novel deep learning approach combined with a synthetic data pipeline for better reflection-transmission separation in real-world scenarios.

Findings

Effective separation of reflections and transmissions in real-world images.

Synthetic data generation improves model training and generalization.

Method outperforms previous techniques on real-world images.

Abstract

The reflections caused by common semi-reflectors, such as glass windows, can impact the performance of computer vision algorithms. State-of-the-art methods can remove reflections on synthetic data and in controlled scenarios. However, they are based on strong assumptions and do not generalize well to real-world images. Contrary to a common misconception, real-world images are challenging even when polarization information is used. We present a deep learning approach to separate the reflected and the transmitted components of the recorded irradiance, which explicitly uses the polarization properties of light. To train it, we introduce an accurate synthetic data generation pipeline, which simulates realistic reflections, including those generated by curved and non-ideal surfaces, non-static scenes, and high-dynamic-range scenes.