Revisiting Human Action Recognition: Personalization vs. Generalization

TL;DR

This paper demonstrates that personalized action recognition systems outperform general models by analyzing inter- and intra-subject variability in MoCap datasets, proposing a two-stage framework that enhances accuracy.

Contribution

It introduces a personalized, two-stage action recognition framework that significantly improves performance over traditional generalization methods.

Findings

Personalized approaches outperform standard cross-validation strategies.

A two-stage framework improves action recognition accuracy.

Standard descriptors and classifiers benefit from personalization.

Abstract

By thoroughly revisiting the classic human action recognition paradigm, this paper aims at proposing a new approach for the design of effective action classification systems. Taking as testbed publicly available three-dimensional (MoCap) action/activity datasets, we analyzed and validated different training/testing strategies. In particular, considering that each human action in the datasets is performed several times by different subjects, we were able to precisely quantify the effect of inter- and intra-subject variability, so as to figure out the impact of several learning approaches in terms of classification performance. The net result is that standard testing strategies consisting in cross-validating the algorithm using typical splits of the data (holdout, k-fold, or one-subject-out) is always outperformed by a "personalization" strategy which learns how a subject is performing an action. In other words, it is advantageous to customize (i.e., personalize) the method to learn the actions carried out by each subject, rather than trying to generalize the actions executions across subjects. Consequently, we finally propose an action recognition framework consisting of a two-stage classification approach where, given a test action, the subject is first identified before the actual recognition of the action takes place. Despite the basic, off-the-shelf descriptors and standard classifiers adopted, we noted a relevant increase in performance with respect to standard state-of-the-art algorithms, so motivating the usage of personalized approaches for designing effective action recognition systems.