DASC: Robust Dense Descriptor for Multi-modal and Multi-spectral Correspondence Estimation

TL;DR

This paper introduces DASC, a robust dense descriptor leveraging self-similarity for multi-modal and multi-spectral correspondence estimation, addressing photometric and geometric variations with high accuracy.

Contribution

The paper proposes a novel adaptive self-correlation descriptor (DASC) and a geometry-invariant version (GI-DASC) for improved multi-modal dense correspondence estimation.

Findings

DASC outperforms existing descriptors in multi-modal correspondence tasks.

GI-DASC effectively handles scale and rotation variations.

The new benchmark demonstrates the robustness of the proposed methods.

Abstract

Establishing dense correspondences between multiple images is a fundamental task in many applications. However, finding a reliable correspondence in multi-modal or multi-spectral images still remains unsolved due to their challenging photometric and geometric variations. In this paper, we propose a novel dense descriptor, called dense adaptive self-correlation (DASC), to estimate multi-modal and multi-spectral dense correspondences. Based on an observation that self-similarity existing within images is robust to imaging modality variations, we define the descriptor with a series of an adaptive self-correlation similarity measure between patches sampled by a randomized receptive field pooling, in which a sampling pattern is obtained using a discriminative learning. The computational redundancy of dense descriptors is dramatically reduced by applying fast edge-aware filtering. Furthermore, in order to address geometric variations including scale and rotation, we propose a geometry-invariant DASC (GI-DASC) descriptor that effectively leverages the DASC through a superpixel-based representation. For a quantitative evaluation of the GI-DASC, we build a novel multi-modal benchmark as varying photometric and geometric conditions. Experimental results demonstrate the outstanding performance of the DASC and GI-DASC in many cases of multi-modal and multi-spectral dense correspondences.