Multi-modal Image Registration for Correlative Microscopy

TL;DR

This paper introduces two novel methods for multi-modal image registration in correlative microscopy, utilizing fiducials and sparse representation to improve alignment between different microscopy modalities.

Contribution

It presents two innovative registration techniques: one based on fiducials and landmarks, and another using sparse coding for image translation, transforming multi-modal registration into a mono-modal problem.

Findings

Fiducial-based method achieves accurate landmark matching.

Sparse representation method effectively predicts images across modalities.

Experimental results demonstrate improved registration accuracy.

Abstract

Correlative microscopy is a methodology combining the functionality of light microscopy with the high resolution of electron microscopy and other microscopy technologies. Image registration for correlative microscopy is quite challenging because it is a multi-modal, multi-scale and multi-dimensional registration problem. In this report, I introduce two methods of image registration for correlative microscopy. The first method is based on fiducials (beads). I generate landmarks from the fiducials and compute the similarity transformation matrix based on three pairs of nearest corresponding landmarks. A least-squares matching process is applied afterwards to further refine the registration. The second method is inspired by the image analogies approach. I introduce the sparse representation model into image analogies. I first train representative image patches (dictionaries) for pre-registered datasets from two different modalities, and then I use the sparse coding technique to transfer a given image to a predicted image from one modality to another based on the learned dictionaries. The final image registration is between the predicted image and the original image corresponding to the given image in the different modality. The method transforms a multi-modal registration problem to a mono-modal one. I test my approaches on Transmission Electron Microscopy (TEM) and confocal microscopy images. Experimental results of the methods are also shown in this report.