DeepCAMP: Deep Convolutional Action & Attribute Mid-Level Patterns

TL;DR

DeepCAMP introduces a novel CNN that mines discriminative mid-level patches for fine-grained human action and attribute recognition, achieving state-of-the-art results without requiring part or pose annotations.

Contribution

It presents a new CNN architecture that iteratively learns and clusters patches, effectively capturing subtle features for action and attribute classification.

Findings

State-of-the-art accuracy on PASCAL VOC 2012 Action dataset

Effective attribute recognition on Berkeley Attributes of People dataset

No need for part or pose annotations

Abstract

The recognition of human actions and the determination of human attributes are two tasks that call for fine-grained classification. Indeed, often rather small and inconspicuous objects and features have to be detected to tell their classes apart. In order to deal with this challenge, we propose a novel convolutional neural network that mines mid-level image patches that are sufficiently dedicated to resolve the corresponding subtleties. In particular, we train a newly de- signed CNN (DeepPattern) that learns discriminative patch groups. There are two innovative aspects to this. On the one hand we pay attention to contextual information in an origi- nal fashion. On the other hand, we let an iteration of feature learning and patch clustering purify the set of dedicated patches that we use. We validate our method for action clas- sification on two challenging datasets: PASCAL VOC 2012 Action and Stanford 40 Actions, and for attribute recogni- tion we use the Berkeley Attributes of People dataset. Our discriminative mid-level mining CNN obtains state-of-the- art results on these datasets, without a need for annotations about parts and poses.