Beyond Binary Contrast: Modeling Continuous Skeleton Action Spaces with Transitional Anchors

TL;DR

TranCLR is a novel contrastive learning framework that models the continuous geometry of human motion in skeleton-based action recognition, improving feature smoothness and class boundary flexibility.

Contribution

It introduces transitional anchors and multi-level geometric calibration to better capture motion continuity in self-supervised skeleton action learning.

Findings

Achieves superior accuracy on NTU RGB+D datasets.

Provides more continuous and uncertainty-aware representations.

Outperforms existing contrastive methods in action recognition.

Abstract

Self-supervised contrastive learning has emerged as a powerful paradigm for skeleton-based action recognition by enforcing consistency in the embedding space. However, existing methods rely on binary contrastive objectives that overlook the intrinsic continuity of human motion, resulting in fragmented feature clusters and rigid class boundaries. To address these limitations, we propose TranCLR, a Transitional anchor-based Contrastive Learning framework that captures the continuous geometry of the action space. Specifically, the proposed Action Transitional Anchor Construction (ATAC) explicitly models the geometric structure of transitional states to enhance the model's perception of motion continuity. Building upon these anchors, a Multi-Level Geometric Manifold Calibration (MGMC) mechanism is introduced to adaptively calibrate the action manifold across multiple levels of continuity, yielding a smoother and more discriminative representation space. Extensive experiments on the NTU RGB+D, NTU RGB+D 120 and PKU-MMD datasets demonstrate that TranCLR achieves superior accuracy and calibration performance, effectively learning continuous and uncertainty-aware skeleton representations. The code is available at https://github.com/Philchieh/TranCLR.