EDM: Equirectangular Projection-Oriented Dense Kernelized Feature Matching

TL;DR

This paper presents EDM, a novel learning-based dense matching algorithm tailored for omnidirectional images, addressing distortions through spherical models and positional embeddings, resulting in significant performance improvements.

Contribution

The first learning-based dense matching method specifically designed for omnidirectional images, incorporating spherical models and geodesic refinement to handle distortions.

Findings

Achieves +26.72 AUC@5° on Matterport3D

Achieves +42.62 AUC@5° on Stanford2D3D

Demonstrates significant performance improvements over existing methods

Abstract

We introduce the first learning-based dense matching algorithm, termed Equirectangular Projection-Oriented Dense Kernelized Feature Matching (EDM), specifically designed for omnidirectional images. Equirectangular projection (ERP) images, with their large fields of view, are particularly suited for dense matching techniques that aim to establish comprehensive correspondences across images. However, ERP images are subject to significant distortions, which we address by leveraging the spherical camera model and geodesic flow refinement in the dense matching method. To further mitigate these distortions, we propose spherical positional embeddings based on 3D Cartesian coordinates of the feature grid. Additionally, our method incorporates bidirectional transformations between spherical and Cartesian coordinate systems during refinement, utilizing a unit sphere to improve matching performance. We demonstrate that our proposed method achieves notable performance enhancements, with improvements of +26.72 and +42.62 in AUC@5{\deg} on the Matterport3D and Stanford2D3D datasets.