Affine steerers for structured keypoint description

TL;DR

This paper introduces affine steerers for deep keypoint descriptors, enabling approximate equivariance to affine transformations, which improves image matching performance on standard benchmarks.

Contribution

It generalizes steerers from rotations to affine transformations using GL(2) representation theory, enhancing control over keypoint description transformations.

Findings

Achieves high control over keypoint description transformations.

Demonstrates improved image matching results.

Attains state-of-the-art performance on benchmarks.

Abstract

We propose a way to train deep learning based keypoint descriptors that makes them approximately equivariant for locally affine transformations of the image plane. The main idea is to use the representation theory of GL(2) to generalize the recently introduced concept of steerers from rotations to affine transformations. Affine steerers give high control over how keypoint descriptions transform under image transformations. We demonstrate the potential of using this control for image matching. Finally, we propose a way to finetune keypoint descriptors with a set of steerers on upright images and obtain state-of-the-art results on several standard benchmarks. Code will be published at github.com/georg-bn/affine-steerers.