Sparse Dictionary-based Attributes for Action Recognition and Summarization

TL;DR

This paper introduces a novel dictionary learning method for action recognition and summarization that maximizes mutual information and employs Gaussian Processes for efficient sparse representation, enabling recognition of both seen and unseen actions.

Contribution

It unifies appearance and class distribution in a mutual information-based objective and uses Gaussian Processes for efficient sparse dictionary learning.

Findings

Effective in recognizing modeled actions.

Generalizes to unseen action categories.

Improves action summarization quality.

Abstract

We present an approach for dictionary learning of action attributes via information maximization. We unify the class distribution and appearance information into an objective function for learning a sparse dictionary of action attributes. The objective function maximizes the mutual information between what has been learned and what remains to be learned in terms of appearance information and class distribution for each dictionary atom. We propose a Gaussian Process (GP) model for sparse representation to optimize the dictionary objective function. The sparse coding property allows a kernel with compact support in GP to realize a very efficient dictionary learning process. Hence we can describe an action video by a set of compact and discriminative action attributes. More importantly, we can recognize modeled action categories in a sparse feature space, which can be generalized to unseen and unmodeled action categories. Experimental results demonstrate the effectiveness of our approach in action recognition and summarization.