Parallel Unlearning in Inherited Model Networks

TL;DR

This paper introduces a parallel unlearning framework for inherited model networks using a DAG structure and Fisher Information Matrix, enabling efficient, one-shot removal of knowledge with minimal computational cost.

Contribution

The paper proposes a novel parallel unlearning method leveraging Fisher Information and DAGs, allowing efficient unlearning in inherited model networks with reduced overhead.

Findings

Achieves complete unlearning with 0% accuracy on unlearned labels.

Maintains high accuracy (94.53%) on retained labels after unlearning.

Accelerates unlearning process by 99% compared to existing methods.

Abstract

Unlearning is challenging in generic learning frameworks with the continuous growth and updates of models exhibiting complex inheritance relationships. This paper presents a novel unlearning framework that enables fully parallel unlearning among models exhibiting inheritance. We use a chronologically Directed Acyclic Graph (DAG) to capture various unlearning scenarios occurring in model inheritance networks. Central to our framework is the Fisher Inheritance Unlearning (FIUn) method, designed to enable efficient parallel unlearning within the DAG. FIUn utilizes the Fisher Information Matrix (FIM) to assess the significance of model parameters for unlearning tasks and adjusts them accordingly. To handle multiple unlearning requests simultaneously, we propose the Merging-FIM (MFIM) function, which consolidates FIMs from multiple upstream models into a unified matrix. This design supports all unlearning scenarios captured by the DAG, enabling one-shot removal of inherited knowledge while significantly reducing computational overhead. Experiments confirm the effectiveness of our unlearning framework. For single-class tasks, it achieves complete unlearning with 0% accuracy for unlearned labels while maintaining 94.53% accuracy for retained labels. For multi-class tasks, the accuracy is 1.07% for unlearned labels and 84.77% for retained labels. Our framework accelerates unlearning by 99% compared to alternative methods. Code is in https://github.com/MJLee00/Parallel-Unlearning-in-Inherited-Model-Networks.