Deep Feature Factorization For Concept Discovery

TL;DR

Deep Feature Factorization (DFF) is a novel method that localizes semantic concepts within images by analyzing hierarchical structures in neural network features, enabling improved co-segmentation and co-localization.

Contribution

The paper introduces DFF, a new technique for visualizing and understanding neural network features through hierarchical clustering and heat maps, achieving state-of-the-art results in co-segmentation and co-localization.

Findings

DFF effectively visualizes semantic concepts in images.

DFF achieves state-of-the-art results on co-segmentation tasks.

Hierarchical clustering reveals meaningful feature structures.

Abstract

We propose Deep Feature Factorization (DFF), a method capable of localizing similar semantic concepts within an image or a set of images. We use DFF to gain insight into a deep convolutional neural network's learned features, where we detect hierarchical cluster structures in feature space. This is visualized as heat maps, which highlight semantically matching regions across a set of images, revealing what the network `perceives' as similar. DFF can also be used to perform co-segmentation and co-localization, and we report state-of-the-art results on these tasks.