Robust Reflection Removal with Flash-only Cues in the Wild

TL;DR

This paper introduces a novel reflection removal method using flash-only cues derived from raw image data, significantly improving accuracy and robustness in real-world scenarios with reflections and misalignments.

Contribution

The work presents a new reflection-free cue from flash-only images and a dedicated model to utilize this cue effectively, outperforming existing methods in reflection removal tasks.

Findings

Outperforms state-of-the-art methods by over 5.23dB in PSNR.

Handles misaligned flash-no-flash pairs with an alignment module, improving performance.

Effective on real-world images with various reflection types.

Abstract

We propose a simple yet effective reflection-free cue for robust reflection removal from a pair of flash and ambient (no-flash) images. The reflection-free cue exploits a flash-only image obtained by subtracting the ambient image from the corresponding flash image in raw data space. The flash-only image is equivalent to an image taken in a dark environment with only a flash on. This flash-only image is visually reflection-free and thus can provide robust cues to infer the reflection in the ambient image. Since the flash-only image usually has artifacts, we further propose a dedicated model that not only utilizes the reflection-free cue but also avoids introducing artifacts, which helps accurately estimate reflection and transmission. Our experiments on real-world images with various types of reflection demonstrate the effectiveness of our model with reflection-free flash-only cues: our model outperforms state-of-the-art reflection removal approaches by more than 5.23dB in PSNR. We extend our approach to handheld photography to address the misalignment between the flash and no-flash pair. With misaligned training data and the alignment module, our aligned model outperforms our previous version by more than 3.19dB in PSNR on a misaligned dataset. We also study using linear RGB images as training data. Our source code and dataset are publicly available at https://github.com/ChenyangLEI/flash-reflection-removal.