Efficient Knowledge Graph Unlearning with Zeroth-order Information

TL;DR

This paper introduces a novel, efficient method for unlearning data from knowledge graphs by approximating influence functions using zeroth-order optimization, significantly reducing computational costs while maintaining high unlearning quality.

Contribution

The paper proposes the first efficient KG unlearning algorithm leveraging zeroth-order optimization and influence function approximation, addressing computational challenges in large-scale graphs.

Findings

Outperforms existing methods in unlearning efficiency

Maintains high unlearning quality with reduced computation

Effective for large-scale knowledge graphs

Abstract

Due to regulations like the Right to be Forgotten, there is growing demand for removing training data and its influence from models. Since full retraining is costly, various machine unlearning methods have been proposed. In this paper, we firstly present an efficient knowledge graph (KG) unlearning algorithm. We remark that KG unlearning is nontrivial due to the distinctive structure of KG and the semantic relations between entities. Also, unlearning by estimating the influence of removed components incurs significant computational overhead when applied to large-scale knowledge graphs. To this end, we define an influence function for KG unlearning and propose to approximate the model's sensitivity without expensive computation of first-order and second-order derivatives for parameter updates. Specifically, we use Taylor expansion to estimate the parameter changes caused by data removal. Given that the first-order gradients and second-order derivatives dominate the computational load, we use the Fisher matrices and zeroth-order optimization to approximate the inverse-Hessian vector product without constructing the computational graphs. Our experimental results demonstrate that the proposed method outperforms other state-of-the-art graph unlearning baselines significantly in terms of unlearning efficiency and unlearning quality. Our code is released at https://github.com/NKUShaw/ZOWFKGIF.