Privacy Drift: Evolving Privacy Concerns in Incremental Learning

TL;DR

This paper introduces 'privacy drift' in federated learning, analyzing how model updates and data shifts affect privacy risks, and provides empirical insights into balancing model performance with privacy preservation.

Contribution

It defines and explores the concept of privacy drift, revealing its impact on privacy risks in federated learning through comprehensive experiments.

Findings

Enhanced model performance can increase privacy risks

Data and concept drift influence privacy leakage

Empirical analysis on CIFAR-100 datasets

Abstract

In the evolving landscape of machine learning (ML), Federated Learning (FL) presents a paradigm shift towards decentralized model training while preserving user data privacy. This paper introduces the concept of ``privacy drift", an innovative framework that parallels the well-known phenomenon of concept drift. While concept drift addresses the variability in model accuracy over time due to changes in the data, privacy drift encapsulates the variation in the leakage of private information as models undergo incremental training. By defining and examining privacy drift, this study aims to unveil the nuanced relationship between the evolution of model performance and the integrity of data privacy. Through rigorous experimentation, we investigate the dynamics of privacy drift in FL systems, focusing on how model updates and data distribution shifts influence the susceptibility of models to privacy attacks, such as membership inference attacks (MIA). Our results highlight a complex interplay between model accuracy and privacy safeguards, revealing that enhancements in model performance can lead to increased privacy risks. We provide empirical evidence from experiments on customized datasets derived from CIFAR-100 (Canadian Institute for Advanced Research, 100 classes), showcasing the impact of data and concept drift on privacy. This work lays the groundwork for future research on privacy-aware machine learning, aiming to achieve a delicate balance between model accuracy and data privacy in decentralized environments.