ArCSEM: Artistic Colorization of SEM Images via Gaussian Splatting

TL;DR

This paper introduces ArCSEM, a method for automatic, high-quality colorization of SEM images by leveraging 3D scene reconstruction and Gaussian Splatting, reducing manual effort and enabling realistic colorized scene synthesis.

Contribution

It presents a novel approach that uses 3D scene representation and Gaussian Splatting to automatically colorize SEM images without manual labeling, facilitating efficient colorization of multiple views.

Findings

Achieves high-quality colorized novel view synthesis of SEM scenes.

No manual intervention needed for 3D scene reconstruction.

Enables automatic full scene or video colorization from few views.

Abstract

Scanning Electron Microscopes (SEMs) are widely renowned for their ability to analyze the surface structures of microscopic objects, offering the capability to capture highly detailed, yet only grayscale, images. To create more expressive and realistic illustrations, these images are typically manually colorized by an artist with the support of image editing software. This task becomes highly laborious when multiple images of a scanned object require colorization. We propose facilitating this process by using the underlying 3D structure of the microscopic scene to propagate the color information to all the captured images, from as little as one colorized view. We explore several scene representation techniques and achieve high-quality colorized novel view synthesis of a SEM scene. In contrast to prior work, there is no manual intervention or labelling involved in obtaining the 3D representation. This enables an artist to color a single or few views of a sequence and automatically retrieve a fully colored scene or video. Project page: https://ronly2460.github.io/ArCSEM