Continual Gesture Learning without Data via Synthetic Feature Sampling

TL;DR

This paper introduces Synthetic Feature Replay, a novel method for data-free continual learning in skeleton-based gesture classification, leveraging synthetic features to improve accuracy without old data.

Contribution

It presents a new approach for data-free class incremental learning in gesture recognition, utilizing synthetic feature sampling to enhance model performance.

Findings

Achieved up to 15% improvement in mean accuracy.

Effectively mitigated accuracy imbalance between classes.

Demonstrated strong generalization of skeleton models to unseen classes.

Abstract

Data-Free Class Incremental Learning (DFCIL) aims to enable models to continuously learn new classes while retraining knowledge of old classes, even when the training data for old classes is unavailable. Although explored primarily with image datasets by researchers, this study focuses on investigating DFCIL for skeleton-based gesture classification due to its significant real-world implications, particularly considering the growing prevalence of VR/AR headsets where gestures serve as the primary means of control and interaction. In this work, we made an intriguing observation: skeleton models trained with base classes(even very limited) demonstrate strong generalization capabilities to unseen classes without requiring additional training. Building on this insight, we developed Synthetic Feature Replay (SFR) that can sample synthetic features from class prototypes to replay for old classes and augment for new classes (under a few-shot setting). Our proposed method showcases significant advancements over the state-of-the-art, achieving up to 15% enhancements in mean accuracy across all steps and largely mitigating the accuracy imbalance between base classes and new classes.