PDLRecover: Privacy-preserving Decentralized Model Recovery with Machine Unlearning

TL;DR

PDLRecover is a privacy-preserving method for efficiently recovering poisoned decentralized models by leveraging historical data and secret sharing, achieving comparable performance to retraining with less cost.

Contribution

It introduces a novel privacy-preserving approach for model recovery in decentralized learning, combining secret sharing and approximate Hessian computation for efficient, secure poisoning mitigation.

Findings

Achieves model recovery with performance close to full retraining.

Reduces computational and time costs significantly.

Ensures privacy by preventing local model leakage.

Abstract

Decentralized learning is vulnerable to poison attacks, where malicious clients manipulate local updates to degrade global model performance. Existing defenses mainly detect and filter malicious models, aiming to prevent a limited number of attackers from corrupting the global model. However, restoring an already compromised global model remains a challenge. A direct approach is to remove malicious clients and retrain the model using only the benign clients. Yet, retraining is time-consuming, computationally expensive, and may compromise model consistency and privacy. We propose PDLRecover, a novel method to recover a poisoned global model efficiently by leveraging historical model information while preserving privacy. The main challenge lies in protecting shared historical models while enabling parameter estimation for model recovery. By exploiting the linearity of approximate Hessian matrix computation, we apply secret sharing to protect historical updates, ensuring local models are not leaked during transmission or reconstruction. PDLRecover introduces client-side preparation, periodic recovery updates, and a final exact update to ensure robustness and convergence of the recovered model. Periodic updates maintain accurate curvature information, and the final step ensures high-quality convergence. Experiments show that the recovered global model achieves performance comparable to a fully retrained model but with significantly reduced computation and time cost. Moreover, PDLRecover effectively prevents leakage of local model parameters, ensuring both accuracy and privacy in recovery.