Deep Multi-Spectral Registration Using Invariant Descriptor Learning

TL;DR

This paper presents a deep learning-based method for aligning cross-spectral images by learning an invariant descriptor, achieving high accuracy in visible to NIR image registration, surpassing existing methods.

Contribution

Introduces a novel deep-learning approach that learns an invariant descriptor for cross-spectral image registration, specifically addressing VIS to NIR alignment challenges.

Findings

Achieves sub-pixel accuracy in cross-spectral registration.

Outperforms existing methods in VIS to NIR alignment.

Uses a patch-metric learned on CIFAR-10 descriptors.

Abstract

In this paper, we introduce a novel deep-learning method to align cross-spectral images. Our approach relies on a learned descriptor which is invariant to different spectra. Multi-modal images of the same scene capture different signals and therefore their registration is challenging and it is not solved by classic approaches. To that end, we developed a feature-based approach that solves the visible (VIS) to Near-Infra-Red (NIR) registration problem. Our algorithm detects corners by Harris and matches them by a patch-metric learned on top of CIFAR-10 network descriptor. As our experiments demonstrate we achieve a high-quality alignment of cross-spectral images with a sub-pixel accuracy. Comparing to other existing methods, our approach is more accurate in the task of VIS to NIR registration.