Incremental Learning with Repetition via Pseudo-Feature Projection

TL;DR

This paper explores incremental learning with repetitive data patterns, proposing a new scenario and a novel method called Horde that dynamically aligns feature extractors, achieving strong results especially with repeated classes.

Contribution

It introduces new incremental learning scenarios with partial repetition, adapts existing methods for fair comparison, and proposes Horde, a novel ensemble approach leveraging class repetition.

Findings

Horde achieves state-of-the-art results with repeated classes.

Exemplar-free methods perform competitively without repetition.

Repetition patterns significantly impact incremental learning strategies.

Abstract

Incremental Learning scenarios do not always represent real-world inference use-cases, which tend to have less strict task boundaries, and exhibit repetition of common classes and concepts in their continual data stream. To better represent these use-cases, new scenarios with partial repetition and mixing of tasks are proposed, where the repetition patterns are innate to the scenario and unknown to the strategy. We investigate how exemplar-free incremental learning strategies are affected by data repetition, and we adapt a series of state-of-the-art approaches to analyse and fairly compare them under both settings. Further, we also propose a novel method (Horde), able to dynamically adjust an ensemble of self-reliant feature extractors, and align them by exploiting class repetition. Our proposed exemplar-free method achieves competitive results in the classic scenario without repetition, and state-of-the-art performance in the one with repetition.