Working hard to know your neighbor's margins: Local descriptor learning loss

TL;DR

This paper proposes a new loss function inspired by Lowe's matching criterion for learning local feature descriptors, achieving state-of-the-art performance with fast computation on various benchmarks.

Contribution

The paper introduces a novel loss that improves local descriptor learning by maximizing the margin between positive and negative patches, outperforming existing regularization methods.

Findings

Achieves state-of-the-art results in stereo, verification, and retrieval tasks.

Produces compact 128-dimensional descriptors comparable to SIFT.

Fast descriptor computation (~1 ms on low-end GPU).

Abstract

We introduce a novel loss for learning local feature descriptors which is inspired by the Lowe's matching criterion for SIFT. We show that the proposed loss that maximizes the distance between the closest positive and closest negative patch in the batch is better than complex regularization methods; it works well for both shallow and deep convolution network architectures. Applying the novel loss to the L2Net CNN architecture results in a compact descriptor -- it has the same dimensionality as SIFT (128) that shows state-of-art performance in wide baseline stereo, patch verification and instance retrieval benchmarks. It is fast, computing a descriptor takes about 1 millisecond on a low-end GPU.