D2HNet: Joint Denoising and Deblurring with Hierarchical Network for Robust Night Image Restoration

TL;DR

This paper introduces D2HNet, a hierarchical neural network that fuses long- and short-exposure images to effectively denoise and deblur night photos, addressing real-world challenges in low-light imaging.

Contribution

The novel D2HNet framework and a two-phase architecture significantly improve night image restoration by reducing domain gap and handling large blur variations.

Findings

Achieves superior visual quality on real night photos

Outperforms existing methods in quantitative scores

Introduces a new synthetic dataset for training and evaluation

Abstract

Night imaging with modern smartphone cameras is troublesome due to low photon count and unavoidable noise in the imaging system. Directly adjusting exposure time and ISO ratings cannot obtain sharp and noise-free images at the same time in low-light conditions. Though many methods have been proposed to enhance noisy or blurry night images, their performances on real-world night photos are still unsatisfactory due to two main reasons: 1) Limited information in a single image and 2) Domain gap between synthetic training images and real-world photos (e.g., differences in blur area and resolution). To exploit the information from successive long- and short-exposure images, we propose a learning-based pipeline to fuse them. A D2HNet framework is developed to recover a high-quality image by deblurring and enhancing a long-exposure image under the guidance of a short-exposure image. To shrink the domain gap, we leverage a two-phase DeblurNet-EnhanceNet architecture, which performs accurate blur removal on a fixed low resolution so that it is able to handle large ranges of blur in different resolution inputs. In addition, we synthesize a D2-Dataset from HD videos and experiment on it. The results on the validation set and real photos demonstrate our methods achieve better visual quality and state-of-the-art quantitative scores. The D2HNet codes and D2-Dataset can be found at https://github.com/zhaoyuzhi/D2HNet.