Lifelong Learning and Selective Forgetting via Contrastive Strategy

TL;DR

This paper introduces a contrastive strategy framework for lifelong learning that enables models to retain useful knowledge while selectively forgetting undesirable information, achieving state-of-the-art results.

Contribution

It proposes a novel contrastive-based approach for learning with selective forgetting, effectively controlling feature distributions for retained and deleted classes.

Findings

Achieves new state-of-the-art performance on four benchmark datasets.

Effectively maintains or disrupts feature distributions for different classes.

Demonstrates the ability to forget specific classes without affecting retained knowledge.

Abstract

Lifelong learning aims to train a model with good performance for new tasks while retaining the capacity of previous tasks. However, some practical scenarios require the system to forget undesirable knowledge due to privacy issues, which is called selective forgetting. The joint task of the two is dubbed Learning with Selective Forgetting (LSF). In this paper, we propose a new framework based on contrastive strategy for LSF. Specifically, for the preserved classes (tasks), we make features extracted from different samples within a same class compacted. And for the deleted classes, we make the features from different samples of a same class dispersed and irregular, i.e., the network does not have any regular response to samples from a specific deleted class as if the network has no training at all. Through maintaining or disturbing the feature distribution, the forgetting and memory of different classes can be or independent of each other. Experiments are conducted on four benchmark datasets, and our method acieves new state-of-the-art.