Immersion and Invariance-based Coding for Privacy-Preserving Federated Learning

TL;DR

This paper proposes a novel privacy-preserving federated learning framework that uses immersion and invariance control theory to encode model parameters, achieving differential privacy without sacrificing accuracy or convergence.

Contribution

It introduces a new control-theoretic approach to embed privacy into federated learning by immersing optimization algorithms into higher-dimensional systems with encoded parameters.

Findings

Achieves differential privacy levels tailored to needs.

Maintains accuracy and convergence rates of standard FL.

Provides a systematic control-based privacy encoding method.

Abstract

Federated learning (FL) has emerged as a method to preserve privacy in collaborative distributed learning. In FL, clients train AI models directly on their devices rather than sharing data with a centralized server, which can pose privacy risks. However, it has been shown that despite FL's partial protection of local data privacy, information about clients' data can still be inferred from shared model updates during training. In recent years, several privacy-preserving approaches have been developed to mitigate this privacy leakage in FL, though they often provide privacy at the cost of model performance or system efficiency. Balancing these trade-offs presents a significant challenge in implementing FL schemes. In this manuscript, we introduce a privacy-preserving FL framework that combines differential privacy and system immersion tools from control theory. The core idea is to treat the optimization algorithms used in standard FL schemes (e.g., gradient-based algorithms) as a dynamical system that we seek to immerse into a higher-dimensional system (referred to as the target optimization algorithm). The target algorithm's dynamics are designed such that, first, the model parameters of the original algorithm are immersed in its parameters; second, it operates on distorted parameters; and third, it converges to an encoded version of the true model parameters from the original algorithm. These encoded parameters can then be decoded at the server to retrieve the original model parameters. We demonstrate that the proposed privacy-preserving scheme can be tailored to offer any desired level of differential privacy for both local and global model parameters, while maintaining the same accuracy and convergence rate as standard FL algorithms.