Descriptor Matching with Convolutional Neural Networks: a Comparison to SIFT

TL;DR

This paper compares CNN-derived features to SIFT descriptors, demonstrating that CNN features outperform SIFT in descriptor matching tasks, even when trained without supervision, indicating a significant advancement in feature matching techniques.

Contribution

The paper provides a comprehensive comparison between CNN features and SIFT descriptors for matching, highlighting CNNs' superior performance in descriptor matching tasks.

Findings

CNN features outperform SIFT in descriptor matching

Unsupervised CNN training still yields strong matching performance

CNN features generalize well across datasets

Abstract

Latest results indicate that features learned via convolutional neural networks outperform previous descriptors on classification tasks by a large margin. It has been shown that these networks still work well when they are applied to datasets or recognition tasks different from those they were trained on. However, descriptors like SIFT are not only used in recognition but also for many correspondence problems that rely on descriptor matching. In this paper we compare features from various layers of convolutional neural nets to standard SIFT descriptors. We consider a network that was trained on ImageNet and another one that was trained without supervision. Surprisingly, convolutional neural networks clearly outperform SIFT on descriptor matching. This paper has been merged with arXiv:1406.6909