A Study on Unsupervised Dictionary Learning and Feature Encoding for Action Classification

TL;DR

This paper evaluates various dictionary learning and feature encoding methods for video action classification, highlighting the effectiveness of sparse coding especially on complex datasets and the importance of encoding mechanisms over dictionary quality.

Contribution

It systematically compares encoding schemes and demonstrates that simple dictionaries can suffice, emphasizing the role of encoding mechanisms in action classification.

Findings

Sparse coding outperforms other methods on complex datasets.

Encoding mechanisms are more critical than dictionary quality.

Random dictionaries can be effective for large datasets.

Abstract

Many efforts have been devoted to develop alternative methods to traditional vector quantization in image domain such as sparse coding and soft-assignment. These approaches can be split into a dictionary learning phase and a feature encoding phase which are often closely connected. In this paper, we investigate the effects of these phases by separating them for video-based action classification. We compare several dictionary learning methods and feature encoding schemes through extensive experiments on KTH and HMDB51 datasets. Experimental results indicate that sparse coding performs consistently better than the other encoding methods in large complex dataset (i.e., HMDB51), and it is robust to different dictionaries. For small simple dataset (i.e., KTH) with less variation, however, all the encoding strategies perform competitively. In addition, we note that the strength of sophisticated encoding approaches comes not from their corresponding dictionaries but the encoding mechanisms, and we can just use randomly selected exemplars as dictionaries for video-based action classification.