Multiple Degradation and Reconstruction Network for Single Image Denoising via Knowledge Distillation

TL;DR

This paper introduces a lightweight, progressive image denoising network that leverages knowledge distillation to achieve high-quality results with fewer parameters, especially under challenging noise conditions.

Contribution

The paper proposes a novel lightweight network architecture and two heterogeneous knowledge distillation strategies for improved single image denoising.

Findings

MDRN outperforms other SID models with fewer parameters.

HMDS enhances tiny models and high-noise denoising performance.

Extensive experiments validate the effectiveness of the proposed methods.

Abstract

Single image denoising (SID) has achieved significant breakthroughs with the development of deep learning. However, the proposed methods are often accompanied by plenty of parameters, which greatly limits their application scenarios. Different from previous works that blindly increase the depth of the network, we explore the degradation mechanism of the noisy image and propose a lightweight Multiple Degradation and Reconstruction Network (MDRN) to progressively remove noise. Meanwhile, we propose two novel Heterogeneous Knowledge Distillation Strategies (HMDS) to enable MDRN to learn richer and more accurate features from heterogeneous models, which make it possible to reconstruct higher-quality denoised images under extreme conditions. Extensive experiments show that our MDRN achieves favorable performance against other SID models with fewer parameters. Meanwhile, plenty of ablation studies demonstrate that the introduced HMDS can improve the performance of tiny models or the model under high noise levels, which is extremely useful for related applications.