ReF -- Rotation Equivariant Features for Local Feature Matching

TL;DR

This paper introduces a novel rotation-equivariant feature extraction method using Steerable E2-CNNs, combined with augmentation-trained CNNs, to enhance local feature matching robustness across various viewing angles and appearance conditions.

Contribution

It proposes a new architecture leveraging steerable CNNs for rotation-specific features and demonstrates their effectiveness when combined with standard CNNs for improved rotation-invariant matching.

Findings

Achieves state-of-the-art rotation robustness on HPatches.

Effective on the new UrbanScenes3D-Air dataset for visual place recognition.

Shows benefits of ensembling and architecture variations.

Abstract

Sparse local feature matching is pivotal for many computer vision and robotics tasks. To improve their invariance to challenging appearance conditions and viewing angles, and hence their usefulness, existing learning-based methods have primarily focused on data augmentation-based training. In this work, we propose an alternative, complementary approach that centers on inducing bias in the model architecture itself to generate `rotation-specific' features using Steerable E2-CNNs, that are then group-pooled to achieve rotation-invariant local features. We demonstrate that this high performance, rotation-specific coverage from the steerable CNNs can be expanded to all rotation angles by combining it with augmentation-trained standard CNNs which have broader coverage but are often inaccurate, thus creating a state-of-the-art rotation-robust local feature matcher. We benchmark our proposed methods against existing techniques on HPatches and a newly proposed UrbanScenes3D-Air dataset for visual place recognition. Furthermore, we present a detailed analysis of the performance effects of ensembling, robust estimation, network architecture variations, and the use of rotation priors.