Multivariate Prototype Representation for Domain-Generalized Incremental Learning

TL;DR

This paper introduces a novel multivariate prototype-based approach for domain-generalized class-incremental learning, effectively addressing catastrophic forgetting and domain shift by adaptively modeling class distributions without storing old exemplars.

Contribution

It proposes a multivariate Normal distribution-based prototype representation that adapts to feature drift and improves class-incremental learning across unseen domains.

Findings

Outperforms existing methods on multiple benchmarks

Effectively models class distribution drift without old exemplars

Enhances robustness to domain shifts in incremental learning

Abstract

Deep learning models suffer from catastrophic forgetting when being fine-tuned with samples of new classes. This issue becomes even more pronounced when faced with the domain shift between training and testing data. In this paper, we study the critical and less explored Domain-Generalized Class-Incremental Learning (DGCIL). We design a DGCIL approach that remembers old classes, adapts to new classes, and can classify reliably objects from unseen domains. Specifically, our loss formulation maintains classification boundaries and suppresses the domain-specific information of each class. With no old exemplars stored, we use knowledge distillation and estimate old class prototype drift as incremental training advances. Our prototype representations are based on multivariate Normal distributions whose means and covariances are constantly adapted to changing model features to represent old classes well by adapting to the feature space drift. For old classes, we sample pseudo-features from the adapted Normal distributions with the help of Cholesky decomposition. In contrast to previous pseudo-feature sampling strategies that rely solely on average mean prototypes, our method excels at capturing varying semantic information. Experiments on several benchmarks validate our claims.