Hybrid multi-layer Deep CNN/Aggregator feature for image classification

TL;DR

This paper introduces a hybrid image classification method combining unsupervised aggregators with deep CNN features, achieving high accuracy with reduced computational cost and smaller feature size.

Contribution

It proposes a novel hybrid approach that integrates traditional unsupervised aggregators with CNN features, reducing training complexity and feature size while maintaining high performance.

Findings

Outperforms standard BoW and is comparable to Fisher vector with smaller features

Significantly outperforms Fisher vectors when including fully connected layers

Achieves performance similar to adapted DCNNs with less training and testing cost

Abstract

Deep Convolutional Neural Networks (DCNN) have established a remarkable performance benchmark in the field of image classification, displacing classical approaches based on hand-tailored aggregations of local descriptors. Yet DCNNs impose high computational burdens both at training and at testing time, and training them requires collecting and annotating large amounts of training data. Supervised adaptation methods have been proposed in the literature that partially re-learn a transferred DCNN structure from a new target dataset. Yet these require expensive bounding-box annotations and are still computationally expensive to learn. In this paper, we address these shortcomings of DCNN adaptation schemes by proposing a hybrid approach that combines conventional, unsupervised aggregators such as Bag-of-Words (BoW), with the DCNN pipeline by treating the output of intermediate layers as densely extracted local descriptors. We test a variant of our approach that uses only intermediate DCNN layers on the standard PASCAL VOC 2007 dataset and show performance significantly higher than the standard BoW model and comparable to Fisher vector aggregation but with a feature that is 150 times smaller. A second variant of our approach that includes the fully connected DCNN layers significantly outperforms Fisher vector schemes and performs comparably to DCNN approaches adapted to Pascal VOC 2007, yet at only a small fraction of the training and testing cost.