Semantic-Enhanced Feature Matching with Learnable Geometric Verification for Cross-Modal Neuron Registration

TL;DR

This paper introduces a deep learning framework that combines semantic-enhanced features and a learnable geometric verification module to improve cross-modal neuron registration accuracy amidst tissue deformation and limited data.

Contribution

It presents a novel hybrid feature descriptor and a learnable geometric verification module, along with a two-stage training strategy for effective cross-modal neuron registration.

Findings

Achieves high registration accuracy in challenging biomedical images

Effectively handles tissue deformation and modality gap

Requires limited annotated data for training

Abstract

Accurately registering in-vivo two-photon and ex-vivo fluorescence micro-optical sectioning tomography images of individual neurons is critical for structure-function analysis in neuroscience. This task is profoundly challenging due to a significant cross-modality appearance gap, the scarcity of annotated data and severe tissue deformations. We propose a novel deep learning framework to address these issues. Our method introduces a semantic-enhanced hybrid feature descriptor, which fuses the geometric precision of local features with the contextual robustness of a vision foundation model DINOV3 to bridge the modality gap. To handle complex deformations, we replace traditional RANSAC with a learnable Geometric Consistency Confidence Module, a novel classifier trained to identify and reject physically implausible correspondences. A data-efficient two-stage training strategy, involving pre-training on synthetically deformed data and fine-tuning on limited real data, overcomes the data scarcity problem. Our framework provides a robust and accurate solution for high-precision registration in challenging biomedical imaging scenarios, enabling large-scale correlative studies.