Automatic Attribute Discovery with Neural Activations

TL;DR

This paper introduces an automatic method leveraging neural activations to discover and analyze visual attributes from noisy web image-text data, providing insights into perceptual depth and enabling semantic region localization.

Contribution

It presents a novel approach that uses neural activation divergences to identify visual attributes without supervised labels, advancing unsupervised attribute discovery from web data.

Findings

Neural activations effectively discover visual attributes from noisy data.

Layered neural network structure reveals perceptual depth of attributes.

Highly-activating neurons help locate semantically relevant image regions.

Abstract

How can a machine learn to recognize visual attributes emerging out of online community without a definitive supervised dataset? This paper proposes an automatic approach to discover and analyze visual attributes from a noisy collection of image-text data on the Web. Our approach is based on the relationship between attributes and neural activations in the deep network. We characterize the visual property of the attribute word as a divergence within weakly-annotated set of images. We show that the neural activations are useful for discovering and learning a classifier that well agrees with human perception from the noisy real-world Web data. The empirical study suggests the layered structure of the deep neural networks also gives us insights into the perceptual depth of the given word. Finally, we demonstrate that we can utilize highly-activating neurons for finding semantically relevant regions.