Video Demoireing with Relation-Based Temporal Consistency

TL;DR

This paper introduces a novel dataset and a deep learning model for removing moire patterns from videos, emphasizing temporal consistency to enhance video quality.

Contribution

The work presents the first hand-held video demoireing dataset and a relation-based temporal consistency loss to improve temporal coherence in video demoireing models.

Findings

The proposed model outperforms existing methods in maintaining temporal consistency.

The dataset enables effective training and evaluation of video demoireing algorithms.

The relation-based loss significantly improves frame-level quality and temporal stability.

Abstract

Moire patterns, appearing as color distortions, severely degrade image and video qualities when filming a screen with digital cameras. Considering the increasing demands for capturing videos, we study how to remove such undesirable moire patterns in videos, namely video demoireing. To this end, we introduce the first hand-held video demoireing dataset with a dedicated data collection pipeline to ensure spatial and temporal alignments of captured data. Further, a baseline video demoireing model with implicit feature space alignment and selective feature aggregation is developed to leverage complementary information from nearby frames to improve frame-level video demoireing. More importantly, we propose a relation-based temporal consistency loss to encourage the model to learn temporal consistency priors directly from ground-truth reference videos, which facilitates producing temporally consistent predictions and effectively maintains frame-level qualities. Extensive experiments manifest the superiority of our model. Code is available at \url{https://daipengwa.github.io/VDmoire_ProjectPage/}.