Direction-aware Video Demoireing with Temporal-guided Bilateral Learning

TL;DR

This paper introduces DTNet, a novel direction-aware and temporal-guided bilateral learning network that effectively removes moire patterns from videos by combining frequency domain analysis, alignment, and color refinement, leading to superior results.

Contribution

The paper proposes a unified framework with two stages, FDDA and TDR, incorporating directional DCT modes and temporal-guided bilateral learning for improved video demoireing.

Findings

Outperforms state-of-the-art methods by 2.3 dB in PSNR

Effectively detects prominent moire edges using directional DCT modes

Preserves color and detail while removing moire patterns

Abstract

Moire patterns occur when capturing images or videos on screens, severely degrading the quality of the captured images or videos. Despite the recent progresses, existing video demoireing methods neglect the physical characteristics and formation process of moire patterns, significantly limiting the effectiveness of video recovery. This paper presents a unified framework, DTNet, a direction-aware and temporal-guided bilateral learning network for video demoireing. DTNet effectively incorporates the process of moire pattern removal, alignment, color correction, and detail refinement. Our proposed DTNet comprises two primary stages: Frame-level Direction-aware Demoireing and Alignment (FDDA) and Tone and Detail Refinement (TDR). In FDDA, we employ multiple directional DCT modes to perform the moire pattern removal process in the frequency domain, effectively detecting the prominent moire edges. Then, the coarse and fine-grained alignment is applied on the demoired features for facilitating the utilization of neighboring information. In TDR, we propose a temporal-guided bilateral learning pipeline to mitigate the degradation of color and details caused by the moire patterns while preserving the restored frequency information in FDDA. Guided by the aligned temporal features from FDDA, the affine transformations for the recovery of the ultimate clean frames are learned in TDR. Extensive experiments demonstrate that our video demoireing method outperforms state-of-the-art approaches by 2.3 dB in PSNR, and also delivers a superior visual experience. Our code is available at https://github.com/rebeccaeexu/DTNet.