Affinity Contrastive Learning for Skeleton-based Human Activity Understanding

TL;DR

This paper proposes ACLNet, a novel affinity contrastive learning framework that leverages class relationships and adaptive strategies to enhance feature discrimination in skeleton-based human activity understanding.

Contribution

Introducing affinity metrics, activity superclasses, and a dynamic temperature schedule to improve contrastive learning for skeleton-based activity recognition.

Findings

Outperforms existing methods on multiple datasets

Improves discrimination of hard positive and negative samples

Enhances performance in action, gait, and re-identification tasks

Abstract

In skeleton-based human activity understanding, existing methods often adopt the contrastive learning paradigm to construct a discriminative feature space. However, many of these approaches fail to exploit the structural inter-class similarities and overlook the impact of anomalous positive samples. In this study, we introduce ACLNet, an Affinity Contrastive Learning Network that explores the intricate clustering relationships among human activity classes to improve feature discrimination. Specifically, we propose an affinity metric to refine similarity measurements, thereby forming activity superclasses that provide more informative contrastive signals. A dynamic temperature schedule is also introduced to adaptively adjust the penalty strength for various superclasses. In addition, we employ a margin-based contrastive strategy to improve the separation of hard positive and negative samples within classes. Extensive experiments on NTU RGB+D 60, NTU RGB+D 120, Kinetics-Skeleton, PKU-MMD, FineGYM, and CASIA-B demonstrate the superiority of our method in skeleton-based action recognition, gait recognition, and person re-identification. The source code is available at https://github.com/firework8/ACLNet.