Feature Representation in Convolutional Neural Networks

TL;DR

This paper investigates the use of CNN feature maps at various layers as generic features for classification, demonstrating that lower-layer features can outperform top-layer activations when used with classifiers like Random Forests and SVMs.

Contribution

It shows that CNN features, especially from lower layers, can be effectively used with traditional classifiers to improve image classification performance.

Findings

Lower-layer CNN features can outperform top-layer features for classification.

Using CNN features with Random Forests and SVMs yields better results than the original CNN.

Partially trained or overfitted CNNs can still produce useful features for classification.

Abstract

Convolutional Neural Networks (CNNs) are powerful models that achieve impressive results for image classification. In addition, pre-trained CNNs are also useful for other computer vision tasks as generic feature extractors. This paper aims to gain insight into the feature aspect of CNN and demonstrate other uses of CNN features. Our results show that CNN feature maps can be used with Random Forests and SVM to yield classification results that outperforms the original CNN. A CNN that is less than optimal (e.g. not fully trained or overfitting) can also extract features for Random Forest/SVM that yield competitive classification accuracy. In contrast to the literature which uses the top-layer activations as feature representation of images for other tasks, using lower-layer features can yield better results for classification.