DD_RoTIR: Dual-Domain Image Registration via Image Translation and Hierarchical Feature-matching

TL;DR

DD_RoTIR is a deep learning model that improves multimodal microscopy image registration by combining image translation, hierarchical feature matching, and rotation-equivariant networks, demonstrating robustness across various datasets.

Contribution

The paper introduces DD_RoTIR, a novel deep learning framework that integrates image translation and hierarchical feature matching for enhanced multimodal image registration.

Findings

Achieves high registration accuracy across multiple microscopy datasets

Demonstrates robustness and applicability in diverse biological imaging scenarios

Outperforms existing methods in multimodal image registration tasks

Abstract

Microscopy images obtained from multiple camera lenses or sensors in biological experiments provide a comprehensive understanding of objects from diverse perspectives. However, using multiple microscope setups increases the risk of misalignment of identical target features across different modalities, making multimodal image registration crucial. In this work, we build upon previous successes in biological image translation (XAcGAN) and mono-modal image registration (RoTIR) to develop a deep learning model, Dual-Domain RoTIR (DD_RoTIR), specifically designed to address these challenges. While GAN-based translation models are often considered inadequate for multimodal image registration, we enhance registration accuracy by employing a feature-matching algorithm based on Transformers and rotation equivariant networks. Additionally, hierarchical feature matching is utilized to tackle the complexities of multimodal image registration. Our results demonstrate that the DD_RoTIR model exhibits strong applicability and robustness across multiple microscopy image datasets.