An Empirical Evaluation of Current Convolutional Architectures' Ability to Manage Nuisance Location and Scale Variability

TL;DR

This paper empirically evaluates how well current CNN architectures handle nuisance variations like location and scale, revealing limitations and proposing improved sampling techniques to enhance performance on benchmark datasets.

Contribution

It provides an empirical assessment of CNNs' ability to marginalize nuisance variability and introduces improved sampling methods for better classification accuracy.

Findings

CNNs are not very effective at marginalizing nuisance variability.

Context significantly impacts CNN classification performance.

Proposed sampling techniques improve end-to-end performance to state-of-the-art levels.

Abstract

We conduct an empirical study to test the ability of Convolutional Neural Networks (CNNs) to reduce the effects of nuisance transformations of the input data, such as location, scale and aspect ratio. We isolate factors by adopting a common convolutional architecture either deployed globally on the image to compute class posterior distributions, or restricted locally to compute class conditional distributions given location, scale and aspect ratios of bounding boxes determined by proposal heuristics. In theory, averaging the latter should yield inferior performance compared to proper marginalization. Yet empirical evidence suggests the converse, leading us to conclude that - at the current level of complexity of convolutional architectures and scale of the data sets used to train them - CNNs are not very effective at marginalizing nuisance variability. We also quantify the effects of context on the overall classification task and its impact on the performance of CNNs, and propose improved sampling techniques for heuristic proposal schemes that improve end-to-end performance to state-of-the-art levels. We test our hypothesis on a classification task using the ImageNet Challenge benchmark and on a wide-baseline matching task using the Oxford and Fischer's datasets.