Image Colorization By Capsule Networks

TL;DR

This paper explores a modified Capsule Network, called ColorCapsNet, for image colorization, leveraging deep features and a new loss function to improve colorization performance.

Contribution

It introduces a novel CapsNet-based architecture for image colorization, integrating deep features and MSE loss, which is a new application of CapsNets.

Findings

Promising results on DIV2K dataset

Effective adaptation of CapsNet for colorization

Potential for further research in capsule-based image processing

Abstract

In this paper, a simple topology of Capsule Network (CapsNet) is investigated for the problem of image colorization. The generative and segmentation capabilities of the original CapsNet topology, which is proposed for image classification problem, is leveraged for the colorization of the images by modifying the network as follows:1) The original CapsNet model is adapted to map the grayscale input to the output in the CIE Lab colorspace, 2) The feature detector part of the model is updated by using deeper feature layers inherited from VGG-19 pre-trained model with weights in order to transfer low-level image representation capability to this model, 3) The margin loss function is modified as Mean Squared Error (MSE) loss to minimize the image-to-imagemapping. The resulting CapsNet model is named as Colorizer Capsule Network (ColorCapsNet).The performance of the ColorCapsNet is evaluated on the DIV2K dataset and promising results are obtained to investigate Capsule Networks further for image colorization problem.