Nanoscale Microscopy Images Colorization Using Neural Networks

TL;DR

This paper presents two neural network-based methods for automatic colorization of grayscale microscopy images, improving accuracy and naturalness by leveraging semantic information and pre-trained models.

Contribution

Introduces two neural network architectures for microscopy image colorization, combining feature extraction and style transfer for precise and natural coloring.

Findings

Accurately colorizes complex microscopy images

Produces natural colors aligned with training data

Demonstrates effectiveness across various microscopy types

Abstract

Microscopy images are powerful tools and widely used in the majority of research areas, such as biology, chemistry, physics and materials fields by various microscopies (scanning electron microscope (SEM), atomic force microscope (AFM) and the optical microscope, et al.). However, most of the microscopy images are colorless due to the unique imaging mechanism. Though investigating on some popular solutions proposed recently about colorizing images, we notice the process of those methods are usually tedious, complicated, and time-consuming. In this paper, inspired by the achievement of machine learning algorithms on different science fields, we introduce two artificial neural networks for gray microscopy image colorization: An end-to-end convolutional neural network (CNN) with a pre-trained model for feature extraction and a pixel-to-pixel neural style transfer convolutional neural network (NST-CNN), which can colorize gray microscopy images with semantic information learned from a user-provided colorful image at inference time. The results demonstrate that our algorithm not only can colorize the microscopy images under complex circumstances precisely but also make the color naturally according to the training of a massive number of nature images with proper hue and saturation.