Improved Multiple-Image-Based Reflection Removal Algorithm Using Deep Neural Networks

TL;DR

This paper introduces a deep neural network-based method for reflection removal in images captured through semi-reflective media, utilizing multiple images, CNNs, GANs, and auto-encoders for improved accuracy and speed.

Contribution

It presents a novel multi-image reflection removal approach combining CNN, GAN, and auto-encoder architectures, significantly enhancing performance and computational efficiency.

Findings

Achieves superior quantitative and qualitative reflection removal results.

Demonstrates faster processing speed than traditional optimization-based methods.

Effectively classifies and regenerates background edges to improve image clarity.

Abstract

When imaging through a semi-reflective medium such as glass, the reflection of another scene can often be found in the captured images. It degrades the quality of the images and affects their subsequent analyses. In this paper, a novel deep neural network approach for solving the reflection problem in imaging is presented. Traditional reflection removal methods not only require long computation time for solving different optimization functions, their performance is also not guaranteed. As array cameras are readily available in nowadays imaging devices, we first suggest in this paper a multiple-image based depth estimation method using a convolutional neural network (CNN). The proposed network avoids the depth ambiguity problem due to the reflection in the image, and directly estimates the depths along the image edges. They are then used to classify the edges as belonging to the background or reflection. Since edges having similar depth values are error prone in the classification, they are removed from the reflection removal process. We suggest a generative adversarial network (GAN) to regenerate the removed background edges. Finally, the estimated background edge map is fed to another auto-encoder network to assist the extraction of the background from the original image. Experimental results show that the proposed reflection removal algorithm achieves superior performance both quantitatively and qualitatively as compared to the state-of-the-art methods. The proposed algorithm also shows much faster speed compared to the existing approaches using the traditional optimization methods.