Bayesian Learning-driven Prototypical Contrastive Loss for Class-Incremental Learning

TL;DR

This paper introduces a Bayesian learning-driven contrastive loss for prototypical networks, enhancing class-incremental learning by effectively reducing catastrophic forgetting and improving class separation.

Contribution

It presents a novel contrastive loss integrated with Bayesian learning to adaptively balance learning objectives in class-incremental scenarios.

Findings

Outperforms state-of-the-art methods on CIFAR-10, CIFAR-100, and ImageNet100 datasets.

Effectively reduces intra-class and increases inter-class distances.

Demonstrates robustness in GNSS interference classification.

Abstract

The primary objective of methods in continual learning is to learn tasks in a sequential manner over time (sometimes from a stream of data), while mitigating the detrimental phenomenon of catastrophic forgetting. This paper proposes a method to learn an effective representation between previous and newly encountered class prototypes. We propose a prototypical network with a Bayesian learning-driven contrastive loss (BLCL), tailored specifically for class-incremental learning scenarios. We introduce a contrastive loss that incorporates novel classes into the latent representation by reducing intra-class and increasing inter-class distance. Our approach dynamically adapts the balance between the cross-entropy and contrastive loss functions with a Bayesian learning technique. Experimental results conducted on the CIFAR-10, CIFAR-100, and ImageNet100 datasets for image classification and images of a GNSS-based dataset for interference classification validate the efficacy of our method, showcasing its superiority over existing state-of-the-art approaches. Git: https://gitlab.cc-asp.fraunhofer.de/darcy_gnss/gnss_class_incremental_learning