Network Comparison Study of Deep Activation Feature Discriminability with Novel Objects

TL;DR

This paper compares the discriminability of deep neural network features across different architectures for novel objects, using manifold analysis on benchmark datasets to inform network selection.

Contribution

It provides a systematic analysis of how different DNN architectures encode novel object features, aiding in better network choice for transfer learning tasks.

Findings

Different architectures focus on different features.

Network discriminability varies across datasets.

Method applicable to any labeled visual dataset.

Abstract

Feature extraction has always been a critical component of the computer vision field. More recently, state-of-the-art computer visions algorithms have incorporated Deep Neural Networks (DNN) in feature extracting roles, creating Deep Convolutional Activation Features (DeCAF). The transferability of DNN knowledge domains has enabled the wide use of pretrained DNN feature extraction for applications with novel object classes, especially those with limited training data. This study analyzes the general discriminability of novel object visual appearances encoded into the DeCAF space of six of the leading visual recognition DNN architectures. The results of this study characterize the Mahalanobis distances and cosine similarities between DeCAF object manifolds across two visual object tracking benchmark data sets. The backgrounds surrounding each object are also included as an object classes in the manifold analysis, providing a wider range of novel classes. This study found that different network architectures led to different network feature focuses that must to be considered in the network selection process. These results are generated from the VOT2015 and UAV123 benchmark data sets; however, the proposed methods can be applied to efficiently compare estimated network performance characteristics for any labeled visual data set.