Mask and Compress: Efficient Skeleton-based Action Recognition in Continual Learning

TL;DR

This paper introduces CHARON, an efficient continual learning framework for skeleton-based action recognition that maintains high accuracy with low computational cost, setting new benchmarks on NTU datasets.

Contribution

We propose CHARON, a novel continual learning method for skeleton-based action recognition that combines sampling, interpolation, and masking for efficiency and improved accuracy.

Findings

CHARON achieves state-of-the-art results on Split NTU-60 and Split NTU-120 datasets.

The framework maintains performance while reducing computational overhead.

Experimental results demonstrate the effectiveness of the proposed techniques.

Abstract

The use of skeletal data allows deep learning models to perform action recognition efficiently and effectively. Herein, we believe that exploring this problem within the context of Continual Learning is crucial. While numerous studies focus on skeleton-based action recognition from a traditional offline perspective, only a handful venture into online approaches. In this respect, we introduce CHARON (Continual Human Action Recognition On skeletoNs), which maintains consistent performance while operating within an efficient framework. Through techniques like uniform sampling, interpolation, and a memory-efficient training stage based on masking, we achieve improved recognition accuracy while minimizing computational overhead. Our experiments on Split NTU-60 and the proposed Split NTU-120 datasets demonstrate that CHARON sets a new benchmark in this domain. The code is available at https://github.com/Sperimental3/CHARON.