Triple-level Model Inferred Collaborative Network Architecture for Video Deraining

TL;DR

This paper introduces TMICS, a triple-level optimization framework that automatically searches for optimal network architectures to improve video deraining, effectively handling diverse rain streaks and enhancing temporal consistency.

Contribution

It proposes a novel triple-level model-guided auto-searching framework combining hyper-parameter optimization, collaborative network structures, and inter-frame modules for video deraining.

Findings

Significant improvements in fidelity over state-of-the-art methods.

Enhanced temporal consistency in derained videos.

Effective handling of diverse rain streak distributions.

Abstract

Video deraining is an important issue for outdoor vision systems and has been investigated extensively. However, designing optimal architectures by the aggregating model formation and data distribution is a challenging task for video deraining. In this paper, we develop a model-guided triple-level optimization framework to deduce network architecture with cooperating optimization and auto-searching mechanism, named Triple-level Model Inferred Cooperating Searching (TMICS), for dealing with various video rain circumstances. In particular, to mitigate the problem that existing methods cannot cover various rain streaks distribution, we first design a hyper-parameter optimization model about task variable and hyper-parameter. Based on the proposed optimization model, we design a collaborative structure for video deraining. This structure includes Dominant Network Architecture (DNA) and Companionate Network Architecture (CNA) that is cooperated by introducing an Attention-based Averaging Scheme (AAS). To better explore inter-frame information from videos, we introduce a macroscopic structure searching scheme that searches from Optical Flow Module (OFM) and Temporal Grouping Module (TGM) to help restore latent frame. In addition, we apply the differentiable neural architecture searching from a compact candidate set of task-specific operations to discover desirable rain streaks removal architectures automatically. Extensive experiments on various datasets demonstrate that our model shows significant improvements in fidelity and temporal consistency over the state-of-the-art works. Source code is available at https://github.com/vis-opt-group/TMICS.