Theoretical Analysis of Privacy Leakage in Trustworthy Federated Learning: A Perspective from Linear Algebra and Optimization Theory

TL;DR

This paper offers a theoretical analysis of privacy leakage in federated learning using linear algebra and optimization theory, identifying conditions to prevent data reconstruction and providing bounds on privacy risks.

Contribution

It introduces a novel theoretical framework analyzing privacy leakage in federated learning from linear algebra and optimization perspectives, with practical conditions for privacy preservation.

Findings

Full-rank Jacobian matrices ensure data privacy.

A sufficient batch size condition prevents data reconstruction.

An upper bound on privacy leakage relates batch size and data distortion.

Abstract

Federated learning has emerged as a promising paradigm for collaborative model training while preserving data privacy. However, recent studies have shown that it is vulnerable to various privacy attacks, such as data reconstruction attacks. In this paper, we provide a theoretical analysis of privacy leakage in federated learning from two perspectives: linear algebra and optimization theory. From the linear algebra perspective, we prove that when the Jacobian matrix of the batch data is not full rank, there exist different batches of data that produce the same model update, thereby ensuring a level of privacy. We derive a sufficient condition on the batch size to prevent data reconstruction attacks. From the optimization theory perspective, we establish an upper bound on the privacy leakage in terms of the batch size, the distortion extent, and several other factors. Our analysis provides insights into the relationship between privacy leakage and various aspects of federated learning, offering a theoretical foundation for designing privacy-preserving federated learning algorithms.