Knowledge Swapping via Learning and Unlearning

TL;DR

This paper introduces Knowledge Swapping, a new task for selectively updating pretrained models by forgetting, retaining, and acquiring knowledge, supported by a novel strategy and extensive experiments across multiple vision tasks.

Contribution

It proposes the Knowledge Swapping task and the Learning Before Forgetting strategy, providing a new framework for controlled knowledge updating in pretrained models.

Findings

Learning Before Forgetting improves knowledge swapping effectiveness.

Forgetting occurs from high-level to low-level features.

Experiments validate strategy across image classification, detection, and segmentation.

Abstract

We introduce \textbf{Knowledge Swapping}, a novel task designed to selectively regulate knowledge of a pretrained model by enabling the forgetting of user\-specified information, retaining essential knowledge, and acquiring new knowledge simultaneously. By delving into the analysis of knock-on feature hierarchy, we find that incremental learning typically progresses from low\-level representations to higher\-level semantics, whereas forgetting tends to occur in the opposite direction\-starting from high-level semantics and moving down to low-level features. Building upon this, we propose to benchmark the knowledge swapping task with the strategy of \textit{Learning Before Forgetting}. Comprehensive experiments on various tasks like image classification, object detection, and semantic segmentation validate the effectiveness of the proposed strategy. The source code is available at \href{https://github.com/xingmingyu123456/KnowledgeSwapping}{https://github.com/xingmingyu123456/KnowledgeSwapping}.