Spectrally Consistent UNet for High Fidelity Image Transformations

TL;DR

This paper introduces Guided UNet (GUNet), a spectrally consistent upsampling method using Guided Image Filter, improving high-fidelity image transformations by reducing artefacts and maintaining spectral integrity.

Contribution

The paper proposes a novel upsampling module based on Guided Image Filter, enhancing UNet architectures with spectral consistency for better image transformation quality.

Findings

GUNet produces higher fidelity outputs in image transformation tasks.

Spectral analysis shows reduced artefacts with the new upsampling method.

GUNet outperforms traditional UNet in inverse tone mapping and colourisation.

Abstract

Convolutional Neural Networks (CNNs) are the current de-facto models used for many imaging tasks due to their high learning capacity as well as their architectural qualities. The ubiquitous UNet architecture provides an efficient and multi-scale solution that combines local and global information. Despite the success of UNet architectures, the use of upsampling layers can cause artefacts. In this work, a method for assessing the structural biases of UNets and the effects these have on the outputs is presented, characterising their impact in the Fourier domain. A new upsampling module is proposed, based on a novel use of the Guided Image Filter, that provides spectrally consistent outputs when used in a UNet architecture, forming the Guided UNet (GUNet). The GUNet architecture is applied and evaluated for example applications of inverse tone mapping/dynamic range expansion and colourisation from grey-scale images and is shown to provide higher fidelity outputs.