When Kernel Methods meet Feature Learning: Log-Covariance Network for Action Recognition from Skeletal Data

TL;DR

This paper introduces a novel Log-Covariance Network that combines kernel methods and feature learning for human action recognition from skeletal data, demonstrating competitive performance with shallow networks.

Contribution

It proposes a hybrid approach that leverages covariance representations with shallow networks, bridging kernel methods and neural networks for improved action recognition.

Findings

Effective modeling of dynamics reduces the need for deep networks.

The approach performs well across six public datasets.

Combines advantages of kernel methods and feature learning.

Abstract

Human action recognition from skeletal data is a hot research topic and important in many open domain applications of computer vision, thanks to recently introduced 3D sensors. In the literature, naive methods simply transfer off-the-shelf techniques from video to the skeletal representation. However, the current state-of-the-art is contended between to different paradigms: kernel-based methods and feature learning with (recurrent) neural networks. Both approaches show strong performances, yet they exhibit heavy, but complementary, drawbacks. Motivated by this fact, our work aims at combining together the best of the two paradigms, by proposing an approach where a shallow network is fed with a covariance representation. Our intuition is that, as long as the dynamics is effectively modeled, there is no need for the classification network to be deep nor recurrent in order to score favorably. We validate this hypothesis in a broad experimental analysis over 6 publicly available datasets.