How to Train Neural Networks for Flare Removal

TL;DR

This paper introduces a novel method for training neural networks to remove lens flare artifacts from photographs by generating semi-synthetic training data based on optical modeling, enabling effective real-world flare removal.

Contribution

It presents a new data synthesis approach based on optical modeling to train neural networks for flare removal, overcoming the lack of training data.

Findings

Models trained with synthetic data generalize well to real flares.

Data synthesis is critical for effective flare removal.

Neural networks can successfully remove diverse lens flares.

Abstract

When a camera is pointed at a strong light source, the resulting photograph may contain lens flare artifacts. Flares appear in a wide variety of patterns (halos, streaks, color bleeding, haze, etc.) and this diversity in appearance makes flare removal challenging. Existing analytical solutions make strong assumptions about the artifact's geometry or brightness, and therefore only work well on a small subset of flares. Machine learning techniques have shown success in removing other types of artifacts, like reflections, but have not been widely applied to flare removal due to the lack of training data. To solve this problem, we explicitly model the optical causes of flare either empirically or using wave optics, and generate semi-synthetic pairs of flare-corrupted and clean images. This enables us to train neural networks to remove lens flare for the first time. Experiments show our data synthesis approach is critical for accurate flare removal, and that models trained with our technique generalize well to real lens flares across different scenes, lighting conditions, and cameras.