Neural Activation Constellations: Unsupervised Part Model Discovery with Convolutional Networks

TL;DR

This paper introduces an unsupervised method to discover object parts by identifying constellations of neural activation patterns in convolutional networks, improving fine-grained recognition without requiring annotations.

Contribution

It presents a novel unsupervised approach for part model discovery using neural activation constellations, applicable to both generic and fine-grained classification tasks.

Findings

Outperforms existing methods on fine-grained recognition datasets without annotations.

Achieves state-of-the-art results on the Stanford Dog dataset.

Neural constellation models enhance data augmentation for fine-tuning.

Abstract

Part models of object categories are essential for challenging recognition tasks, where differences in categories are subtle and only reflected in appearances of small parts of the object. We present an approach that is able to learn part models in a completely unsupervised manner, without part annotations and even without given bounding boxes during learning. The key idea is to find constellations of neural activation patterns computed using convolutional neural networks. In our experiments, we outperform existing approaches for fine-grained recognition on the CUB200-2011, NA birds, Oxford PETS, and Oxford Flowers dataset in case no part or bounding box annotations are available and achieve state-of-the-art performance for the Stanford Dog dataset. We also show the benefits of neural constellation models as a data augmentation technique for fine-tuning. Furthermore, our paper unites the areas of generic and fine-grained classification, since our approach is suitable for both scenarios. The source code of our method is available online at http://www.inf-cv.uni-jena.de/part_discovery