Let's See Clearly: Contaminant Artifact Removal for Moving Cameras

TL;DR

This paper introduces a novel video restoration method that automatically detects and removes lens contaminants like dust and moisture, leveraging attention maps, flow completion, and multi-frame processing to produce clear, consistent videos.

Contribution

It presents a new framework combining attention detection, flow hallucination, and recurrent restoration, trained on a synthetic dataset, to effectively remove diverse lens contaminants.

Findings

Outperforms existing methods qualitatively and quantitatively

Effectively removes various contaminants from videos

Ensures temporal consistency across frames

Abstract

Contaminants such as dust, dirt and moisture adhering to the camera lens can greatly affect the quality and clarity of the resulting image or video. In this paper, we propose a video restoration method to automatically remove these contaminants and produce a clean video. Our approach first seeks to detect attention maps that indicate the regions that need to be restored. In order to leverage the corresponding clean pixels from adjacent frames, we propose a flow completion module to hallucinate the flow of the background scene to the attention regions degraded by the contaminants. Guided by the attention maps and completed flows, we propose a recurrent technique to restore the input frame by fetching clean pixels from adjacent frames. Finally, a multi-frame processing stage is used to further process the entire video sequence in order to enforce temporal consistency. The entire network is trained on a synthetic dataset that approximates the physical lighting properties of contaminant artifacts. This new dataset and our novel framework lead to our method that is able to address different contaminants and outperforms competitive restoration approaches both qualitatively and quantitatively.