Astronomical Image Colorization and upscaling with Generative Adversarial Networks

TL;DR

This paper presents a GAN-based automated method for colorizing and upscaling astronomical images, leveraging transfer learning with ResNet-18, and evaluates its performance using quantitative metrics like FID, L1, and L2 distances.

Contribution

It introduces a novel application of GANs for astronomical image colorization and super-resolution using transfer learning and compares different color spaces for improved results.

Findings

Generated images are visually appealing with high-resolution hallucinations.

Quantitative evaluation shows effective colorization and upscaling performance.

Model outperforms baseline metrics in FID, L1, and L2 distances.

Abstract

Automatic colorization of images without human intervention has been a subject of interest in the machine learning community for a brief period of time. Assigning color to an image is a highly ill-posed problem because of its innate nature of possessing very high degrees of freedom; given an image, there is often no single color-combination that is correct. Besides colorization, another problem in reconstruction of images is Single Image Super Resolution, which aims at transforming low resolution images to a higher resolution. This research aims to provide an automated approach for the problem by focusing on a very specific domain of images, namely astronomical images, and process them using Generative Adversarial Networks (GANs). We explore the usage of various models in two different color spaces, RGB and L*a*b. We use transferred learning owing to a small data set, using pre-trained ResNet-18 as a backbone, i.e. encoder for the U-net and fine-tune it further. The model produces visually appealing images which hallucinate high resolution, colorized data in these results which does not exist in the original image. We present our results by evaluating the GANs quantitatively using distance metrics such as L1 distance and L2 distance in each of the color spaces across all channels to provide a comparative analysis. We use Frechet inception distance (FID) to compare the distribution of the generated images with the distribution of the real image to assess the model's performance.