Unsupervised Video Class-Incremental Learning via Deep Embedded Clustering Management

TL;DR

This paper introduces an unsupervised video class-incremental learning method that builds deep clusters from video features without using labels, enabling effective learning without forgetting across multiple datasets.

Contribution

It proposes a novel unsupervised approach that leverages deep embedded clustering for incremental learning of video classes without relying on labels or task boundaries.

Findings

Outperforms baseline methods on UCF101, HMDB51, and Something-to-Something V2 datasets.

Effectively manages video class-incremental learning without supervision.

Demonstrates robustness across multiple standard datasets.

Abstract

Unsupervised video class incremental learning (uVCIL) represents an important learning paradigm for learning video information without forgetting, and without considering any data labels. Prior approaches have focused on supervised class-incremental learning, relying on using the knowledge of labels and task boundaries, which is costly, requires human annotation, or is simply not a realistic option. In this paper, we propose a simple yet effective approach to address the uVCIL. We first consider a deep feature extractor network, providing a set of representative video features during each task without assuming any class or task information. We then progressively build a series of deep clusters from the extracted features. During the successive task learning, the model updated from the previous task is used as an initial state in order to transfer knowledge to the current learning task. We perform in-depth evaluations on three standard video action recognition datasets, including UCF101, HMDB51, and Something-to-Something V2, by ignoring the labels from the supervised setting. Our approach significantly outperforms other baselines on all datasets.