Mid-level Deep Pattern Mining

TL;DR

This paper introduces a novel method for discovering mid-level visual elements by applying pattern mining to CNN activations, leading to more effective scene and object classification with fewer elements.

Contribution

It proposes a new approach combining CNN features with pattern mining for visual element discovery, outperforming previous methods in classification tasks.

Findings

Outperforms previous mid-level visual element discovery methods.

Uses fewer elements to achieve better classification accuracy.

CNN activations from the fully-connected layer are effective for pattern mining.

Abstract

Mid-level visual element discovery aims to find clusters of image patches that are both representative and discriminative. In this work, we study this problem from the prospective of pattern mining while relying on the recently popularized Convolutional Neural Networks (CNNs). Specifically, we find that for an image patch, activations extracted from the first fully-connected layer of CNNs have two appealing properties which enable its seamless integration with pattern mining. Patterns are then discovered from a large number of CNN activations of image patches through the well-known association rule mining. When we retrieve and visualize image patches with the same pattern, surprisingly, they are not only visually similar but also semantically consistent. We apply our approach to scene and object classification tasks, and demonstrate that our approach outperforms all previous works on mid-level visual element discovery by a sizeable margin with far fewer elements being used. Our approach also outperforms or matches recent works using CNN for these tasks. Source code of the complete system is available online.