vCLIMB: A Novel Video Class Incremental Learning Benchmark
Andr\'es Villa, Kumail Alhamoud, Juan Le\'on Alc\'azar, Fabian Caba, Heilbron, Victor Escorcia, Bernard Ghanem

TL;DR
This paper introduces vCLIMB, a new standardized benchmark for video continual learning that addresses class imbalance and data sampling challenges, and proposes a temporal consistency regularization to improve model performance.
Contribution
The paper presents vCLIMB, a novel video continual learning benchmark, and proposes a temporal consistency regularization method to enhance learning in untrimmed video data.
Findings
Significant performance improvement with up to 24% gain on untrimmed tasks.
Identification of unique challenges in video CL such as frame-level instance selection.
Demonstration of the effectiveness of temporal regularization in addressing data sampling issues.
Abstract
Continual learning (CL) is under-explored in the video domain. The few existing works contain splits with imbalanced class distributions over the tasks, or study the problem in unsuitable datasets. We introduce vCLIMB, a novel video continual learning benchmark. vCLIMB is a standardized test-bed to analyze catastrophic forgetting of deep models in video continual learning. In contrast to previous work, we focus on class incremental continual learning with models trained on a sequence of disjoint tasks, and distribute the number of classes uniformly across the tasks. We perform in-depth evaluations of existing CL methods in vCLIMB, and observe two unique challenges in video data. The selection of instances to store in episodic memory is performed at the frame level. Second, untrimmed training data influences the effectiveness of frame sampling strategies. We address these two challenges…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDomain Adaptation and Few-Shot Learning · Multimodal Machine Learning Applications
