Variational Contrastive Learning for Skeleton-based Action Recognition

TL;DR

This paper introduces a variational contrastive learning framework for skeleton-based action recognition, combining probabilistic modeling with contrastive learning to improve representation quality and generalization across datasets.

Contribution

It presents a novel variational contrastive learning approach that captures motion variability and uncertainty, outperforming existing methods especially in low-label scenarios.

Findings

Outperforms existing methods on three benchmarks

Provides more relevant features focusing on key skeleton joints

Shows robustness across different datasets and supervision levels

Abstract

In recent years, self-supervised representation learning for skeleton-based action recognition has advanced with the development of contrastive learning methods. However, most of contrastive paradigms are inherently discriminative and often struggle to capture the variability and uncertainty intrinsic to human motion. To address this issue, we propose a variational contrastive learning framework that integrates probabilistic latent modeling with contrastive self-supervised learning. This formulation enables the learning of structured and semantically meaningful representations that generalize across different datasets and supervision levels. Extensive experiments on three widely used skeleton-based action recognition benchmarks show that our proposed method consistently outperforms existing approaches, particularly in low-label regimes. Moreover, qualitative analyses show that the features provided by our method are more relevant given the motion and sample characteristics, with more focus on important skeleton joints, when compared to the other methods.