CNN Features off-the-shelf: an Astounding Baseline for Recognition

TL;DR

This paper demonstrates that features extracted from off-the-shelf convolutional neural networks serve as a powerful, generic image representation, achieving superior results across diverse recognition tasks without task-specific tuning.

Contribution

It provides extensive experimental evidence that generic CNN features outperform specialized methods in various recognition tasks using simple classifiers.

Findings

CNN features outperform state-of-the-art methods in recognition tasks

Features extracted from pre-trained CNNs are highly versatile

Simple classifiers with CNN features achieve strong results

Abstract

Recent results indicate that the generic descriptors extracted from the convolutional neural networks are very powerful. This paper adds to the mounting evidence that this is indeed the case. We report on a series of experiments conducted for different recognition tasks using the publicly available code and model of the \overfeat network which was trained to perform object classification on ILSVRC13. We use features extracted from the \overfeat network as a generic image representation to tackle the diverse range of recognition tasks of object image classification, scene recognition, fine grained recognition, attribute detection and image retrieval applied to a diverse set of datasets. We selected these tasks and datasets as they gradually move further away from the original task and data the \overfeat network was trained to solve. Astonishingly, we report consistent superior results compared to the highly tuned state-of-the-art systems in all the visual classification tasks on various datasets. For instance retrieval it consistently outperforms low memory footprint methods except for sculptures dataset. The results are achieved using a linear SVM classifier (or $L2$ distance in case of retrieval) applied to a feature representation of size 4096 extracted from a layer in the net. The representations are further modified using simple augmentation techniques e.g. jittering. The results strongly suggest that features obtained from deep learning with convolutional nets should be the primary candidate in most visual recognition tasks.