Re-Evaluating Privacy in Centralized and Decentralized Learning: An Information-Theoretical and Empirical Study

TL;DR

This paper provides an information-theoretical and empirical analysis of privacy in federated learning, revealing that decentralized federated learning often offers better privacy than centralized methods, especially without a trusted server.

Contribution

It introduces a formal mutual information framework to evaluate privacy leakage in federated learning and compares privacy-preserving techniques across centralized and decentralized setups.

Findings

DFL generally offers stronger privacy than CFL in practical scenarios.

Secure Aggregation enhances privacy in both CFL and DFL.

Previous assumptions about privacy in DFL may underestimate information leakage.

Abstract

Decentralized Federated Learning (DFL) has garnered attention for its robustness and scalability compared to Centralized Federated Learning (CFL). While DFL is commonly believed to offer privacy advantages due to the decentralized control of sensitive data, recent work by Pasquini et, al. challenges this view, demonstrating that DFL does not inherently improve privacy against empirical attacks under certain assumptions. For investigating fully this issue, a formal theoretical framework is required. Our study offers a novel perspective by conducting a rigorous information-theoretical analysis of privacy leakage in FL using mutual information. We further investigate the effectiveness of privacy-enhancing techniques like Secure Aggregation (SA) in both CFL and DFL. Our simulations and real-world experiments show that DFL generally offers stronger privacy preservation than CFL in practical scenarios where a fully trusted server is not available. We address discrepancies in previous research by highlighting limitations in their assumptions about graph topology and privacy attacks, which inadequately capture information leakage in FL.