Automatic Registration of SHG and H&E Images with Feature-based Initial Alignment and Intensity-based Instance Optimization: Contribution to the COMULIS Challenge

TL;DR

This paper presents a training-free, multi-modal image registration method combining feature matching and intensity optimization to align SHG microscopy images with H&E slides, demonstrating high success rates and low registration error.

Contribution

The proposed approach introduces a novel, training-free registration pipeline for multi-modal microscopy images, enhancing accuracy and generalizability in challenging tissue imaging scenarios.

Findings

88% success rate in initial alignment

Average registration error of 2.48 units

Method is openly available and integrated into DeeperHistReg

Abstract

The automatic registration of noninvasive second-harmonic generation microscopy to hematoxylin and eosin slides is a highly desired, yet still unsolved problem. The task is challenging because the second-harmonic images contain only partial information, in contrast to the stained H&E slides that provide more information about the tissue morphology. Moreover, both imaging methods have different intensity distributions. Therefore, the task can be formulated as a multi-modal registration problem with missing data. In this work, we propose a method based on automatic keypoint matching followed by deformable registration based on instance optimization. The method does not require any training and is evaluated using the dataset provided in the Learn2Reg challenge by the COMULIS organization. The method achieved relatively good generalizability resulting in 88% of success rate in the initial alignment and average target registration error equal to 2.48 on the external validation set. We openly release the source code and incorporate it in the DeeperHistReg image registration framework.