Fracking Deep Convolutional Image Descriptors

TL;DR

This paper introduces a discriminative deep learning framework for local image descriptors using a siamese CNN architecture with a novel training strategy called 'fracking', achieving significant performance improvements over traditional and existing learned descriptors.

Contribution

It presents a new training approach for siamese CNNs with aggressive mining, leading to superior local image descriptor performance.

Findings

Achieves up to 2.5x better than SIFT in AUC

Outperforms previous state-of-the-art learned descriptors

Demonstrates large performance gains through hyper-parameter tuning

Abstract

In this paper we propose a novel framework for learning local image descriptors in a discriminative manner. For this purpose we explore a siamese architecture of Deep Convolutional Neural Networks (CNN), with a Hinge embedding loss on the L2 distance between descriptors. Since a siamese architecture uses pairs rather than single image patches to train, there exist a large number of positive samples and an exponential number of negative samples. We propose to explore this space with a stochastic sampling of the training set, in combination with an aggressive mining strategy over both the positive and negative samples which we denote as "fracking". We perform a thorough evaluation of the architecture hyper-parameters, and demonstrate large performance gains compared to both standard CNN learning strategies, hand-crafted image descriptors like SIFT, and the state-of-the-art on learned descriptors: up to 2.5x vs SIFT and 1.5x vs the state-of-the-art in terms of the area under the curve (AUC) of the Precision-Recall curve.