LMQFormer: A Laplace-Prior-Guided Mask Query Transformer for Lightweight Snow Removal

TL;DR

This paper introduces LMQFormer, a lightweight transformer-based model for snow removal that effectively uses a Laplace-VQVAE generated mask to reduce parameters and computational costs while achieving state-of-the-art results.

Contribution

The paper proposes a novel Laplace-VQVAE and Mask Query Transformer architecture that significantly reduces model size and computation in snow removal tasks while maintaining high performance.

Findings

Achieves state-of-the-art snow removal quality.

Reduces model parameters and runtime significantly.

Effectively utilizes coarse masks for efficient learning.

Abstract

Snow removal aims to locate snow areas and recover clean images without repairing traces. Unlike the regularity and semitransparency of rain, snow with various patterns and degradations seriously occludes the background. As a result, the state-of-the-art snow removal methods usually retains a large parameter size. In this paper, we propose a lightweight but high-efficient snow removal network called Laplace Mask Query Transformer (LMQFormer). Firstly, we present a Laplace-VQVAE to generate a coarse mask as prior knowledge of snow. Instead of using the mask in dataset, we aim at reducing both the information entropy of snow and the computational cost of recovery. Secondly, we design a Mask Query Transformer (MQFormer) to remove snow with the coarse mask, where we use two parallel encoders and a hybrid decoder to learn extensive snow features under lightweight requirements. Thirdly, we develop a Duplicated Mask Query Attention (DMQA) that converts the coarse mask into a specific number of queries, which constraint the attention areas of MQFormer with reduced parameters. Experimental results in popular datasets have demonstrated the efficiency of our proposed model, which achieves the state-of-the-art snow removal quality with significantly reduced parameters and the lowest running time.