Removing Reflections from RAW Photos
Eric Kee, Adam Pikielny, Kevin Blackburn-Matzen, Marc Levoy

TL;DR
This paper presents a novel system for removing reflections from RAW photos using synthetic training data, optional contextual images, and a two-stage neural network, achieving state-of-the-art results efficiently.
Contribution
The authors introduce a reflection removal system trained solely on synthetic RAW data, utilizing an optional context photo for disambiguation, and demonstrate superior performance over prior methods.
Findings
Achieves state-of-the-art reflection removal on real RAW photos.
Training on synthetic data improves performance more than architectural changes.
Operates efficiently at 256p with fast up-sampling to full resolution.
Abstract
We describe a system to remove real-world reflections from images for consumer photography. Our system operates on linear (RAW) photos, and accepts an optional contextual photo looking in the opposite direction (e.g., the "selfie" camera on a mobile device). This optional photo disambiguates what should be considered the reflection. The system is trained solely on synthetic mixtures of real RAW photos, which we combine using a reflection simulation that is photometrically and geometrically accurate. Our system comprises a base model that accepts the captured photo and optional context photo as input, and runs at 256p, followed by an up-sampling model that transforms 256p images to full resolution. The system produces preview images at 1K in 4.5-6.5s on a MacBook or iPhone 14 Pro. We show SOTA results on RAW photos that were captured in the field to embody typical consumer photos, and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsImage Processing and 3D Reconstruction
MethodsBalanced Selection
