Contextual colorization and denoising for low-light ultra high resolution sequences

TL;DR

This paper introduces an unpaired-learning, multiscale patch-based CycleGAN approach for simultaneous colorization and denoising of low-light ultra high resolution sequences, effectively reducing noise and flicker artifacts.

Contribution

It presents a novel multiscale patch framework and adaptive temporal smoothing for low-light sequence enhancement without requiring ground truth data.

Findings

Outperforms existing methods in subjective quality

Robust to brightness and noise variations

Effectively reduces flickering artifacts

Abstract

Low-light image sequences generally suffer from spatio-temporal incoherent noise, flicker and blurring of moving objects. These artefacts significantly reduce visual quality and, in most cases, post-processing is needed in order to generate acceptable quality. Most state-of-the-art enhancement methods based on machine learning require ground truth data but this is not usually available for naturally captured low light sequences. We tackle these problems with an unpaired-learning method that offers simultaneous colorization and denoising. Our approach is an adaptation of the CycleGAN structure. To overcome the excessive memory limitations associated with ultra high resolution content, we propose a multiscale patch-based framework, capturing both local and contextual features. Additionally, an adaptive temporal smoothing technique is employed to remove flickering artefacts. Experimental results show that our method outperforms existing approaches in terms of subjective quality and that it is robust to variations in brightness levels and noise.