Attention-based Stylisation for Exemplar Image Colourisation

TL;DR

This paper introduces an end-to-end attention-based neural network for exemplar image colourisation that simplifies the process, improves robustness, and produces high-quality, visually appealing colourised images efficiently.

Contribution

It presents a novel unified architecture with integrated attention modules and axial attention, reducing complexity and enhancing robustness in exemplar image colourisation.

Findings

Produces high-quality, visually appealing colourisations.

Reduces system complexity compared to state-of-the-art methods.

Demonstrates efficiency and robustness through experimental validation.

Abstract

Exemplar-based colourisation aims to add plausible colours to a grayscale image using the guidance of a colour reference image. Most of the existing methods tackle the task as a style transfer problem, using a convolutional neural network (CNN) to obtain deep representations of the content of both inputs. Stylised outputs are then obtained by computing similarities between both feature representations in order to transfer the style of the reference to the content of the target input. However, in order to gain robustness towards dissimilar references, the stylised outputs need to be refined with a second colourisation network, which significantly increases the overall system complexity. This work reformulates the existing methodology introducing a novel end-to-end colourisation network that unifies the feature matching with the colourisation process. The proposed architecture integrates attention modules at different resolutions that learn how to perform the style transfer task in an unsupervised way towards decoding realistic colour predictions. Moreover, axial attention is proposed to simplify the attention operations and to obtain a fast but robust cost-effective architecture. Experimental validations demonstrate efficiency of the proposed methodology which generates high quality and visual appealing colourisation. Furthermore, the complexity of the proposed methodology is reduced compared to the state-of-the-art methods.