# SOSNet: Second Order Similarity Regularization for Local Descriptor Learning

## TL;DR

This paper introduces SOSR, a novel regularization method leveraging second order similarity for local descriptor learning, achieving state-of-the-art results across various benchmarks.

## Contribution

The paper proposes SOSR, a new regularization term based on second order similarity, to improve local descriptor learning performance.

## Key findings

- Achieves state-of-the-art performance on multiple benchmarks.
- Demonstrates the effectiveness of SOSR through extensive experiments.
- Links descriptor space utilization to matching accuracy using a von Mises-Fischer distribution.

## Abstract

Despite the fact that Second Order Similarity (SOS) has been used with significant success in tasks such as graph matching and clustering, it has not been exploited for learning local descriptors. In this work, we explore the potential of SOS in the field of descriptor learning by building upon the intuition that a positive pair of matching points should exhibit similar distances with respect to other points in the embedding space. Thus, we propose a novel regularization term, named Second Order Similarity Regularization (SOSR), that follows this principle. By incorporating SOSR into training, our learned descriptor achieves state-of-the-art performance on several challenging benchmarks containing distinct tasks ranging from local patch retrieval to structure from motion. Furthermore, by designing a von Mises-Fischer distribution based evaluation method, we link the utilization of the descriptor space to the matching performance, thus demonstrating the effectiveness of our proposed SOSR. Extensive experimental results, empirical evidence, and in-depth analysis are provided, indicating that SOSR can significantly boost the matching performance of the learned descriptor.