Video Demoireing using Focused-Defocused Dual-Camera System

TL;DR

This paper introduces a dual-camera system capturing synchronized focused and defocused videos to effectively distinguish and remove moire patterns, improving demoireing performance while maintaining tonal and temporal consistency.

Contribution

The novel dual-camera framework leverages defocused videos to guide demoireing of focused videos, addressing challenges in pattern discrimination and coherence preservation.

Findings

Outperforms state-of-the-art demoireing methods

Effectively distinguishes moire from real textures

Maintains tonal and temporal consistency

Abstract

Moire patterns, unwanted color artifacts in images and videos, arise from the interference between spatially high-frequency scene contents and the spatial discrete sampling of digital cameras. Existing demoireing methods primarily rely on single-camera image/video processing, which faces two critical challenges: 1) distinguishing moire patterns from visually similar real textures, and 2) preserving tonal consistency and temporal coherence while removing moire artifacts. To address these issues, we propose a dual-camera framework that captures synchronized videos of the same scene: one in focus (retaining high-quality textures but may exhibit moire patterns) and one defocused (with significantly reduced moire patterns but blurred textures). We use the defocused video to help distinguish moire patterns from real texture, so as to guide the demoireing of the focused video. We propose a frame-wise demoireing pipeline, which begins with an optical flow based alignment step to address any discrepancies in displacement and occlusion between the focused and defocused frames. Then, we leverage the aligned defocused frame to guide the demoireing of the focused frame using a multi-scale CNN and a multi-dimensional training loss. To maintain tonal and temporal consistency, our final step involves a joint bilateral filter to leverage the demoireing result from the CNN as the guide to filter the input focused frame to obtain the final output. Experimental results demonstrate that our proposed framework largely outperforms state-of-the-art image and video demoireing methods.