Physics-Informed Video Flare Synthesis and Removal Leveraging Motion Independence between Flare and Scene

TL;DR

This paper introduces a physics-informed approach for synthesizing and removing lens flare in videos by modeling motion independence, utilizing optical flow, attention mechanisms, and long-range spatio-temporal dependencies, and provides a new dataset for evaluation.

Contribution

It presents the first video flare synthesis and removal method leveraging motion independence, along with a new dataset and a novel neural network architecture for improved performance.

Findings

Outperforms existing methods on synthetic and real videos

Effectively removes dynamic flares while preserving scene details

Reduces flicker and artifacts in flare removal

Abstract

Lens flare is a degradation phenomenon caused by strong light sources. Existing researches on flare removal have mainly focused on images, while the spatiotemporal characteristics of video flare remain largely unexplored. Video flare synthesis and removal pose significantly greater challenges than in image, owing to the complex and mutually independent motion of flare, light sources, and scene content. This motion independence further affects restoration performance, often resulting in flicker and artifacts. To address this issue, we propose a physics-informed dynamic flare synthesis pipeline, which simulates light source motion using optical flow and models the temporal behaviors of both scattering and reflective flares. Meanwhile, we design a video flare removal network that employs an attention module to spatially suppress flare regions and incorporates a Mamba-based temporal modeling component to capture long range spatio-temporal dependencies. This motion-independent spatiotemporal representation effectively eliminates the need for multi-frame alignment, alleviating temporal aliasing between flares and scene content and thereby improving video flare removal performance. Building upon this, we construct the first video flare dataset to comprehensively evaluate our method, which includes a large set of synthetic paired videos and additional real-world videos collected from the Internet to assess generalization capability. Extensive experiments demonstrate that our method consistently outperforms existing video-based restoration and image-based flare removal methods on both real and synthetic videos, effectively removing dynamic flares while preserving light source integrity and maintaining spatiotemporal consistency of scene.