Siamese Machine Unlearning with Knowledge Vaporization and Concentration

TL;DR

This paper introduces a novel Siamese network-based approach for machine unlearning that efficiently removes specific data knowledge without extra memory or full dataset access, improving privacy and utility.

Contribution

It proposes the concepts of knowledge vaporization and concentration, and develops an efficient Siamese unlearning method that overcomes limitations of existing techniques.

Findings

Effective removal of knowledge from forgetting data

Improved model utility on remaining data

Reduced susceptibility to membership inference attacks

Abstract

In response to the practical demands of the ``right to be forgotten" and the removal of undesired data, machine unlearning emerges as an essential technique to remove the learned knowledge of a fraction of data points from trained models. However, existing methods suffer from limitations such as insufficient methodological support, high computational complexity, and significant memory demands. In this work, we propose the concepts of knowledge vaporization and concentration to selectively erase learned knowledge from specific data points while maintaining representations for the remaining data. Utilizing the Siamese networks, we exemplify the proposed concepts and develop an efficient method for machine unlearning. Our proposed Siamese unlearning method does not require additional memory overhead and full access to the remaining dataset. Extensive experiments conducted across multiple unlearning scenarios showcase the superiority of Siamese unlearning over baseline methods, illustrating its ability to effectively remove knowledge from forgetting data, enhance model utility on remaining data, and reduce susceptibility to membership inference attacks.