Secure Distributed/Federated Learning: Prediction-Privacy Trade-Off for Multi-Agent System

TL;DR

This paper explores the balance between privacy and accuracy in decentralized federated learning systems with limited agent capabilities, proposing an optimization framework and algorithm to enhance privacy while maintaining learning effectiveness.

Contribution

It introduces a novel privacy-aware assignment scheme for multi-agent federated learning that accounts for agent constraints and optimizes the privacy-accuracy trade-off.

Findings

The proposed algorithm converges to a self-consistent solution.

Numerical results demonstrate effective privacy preservation.

The approach balances privacy and prediction accuracy in distributed settings.

Abstract

Decentralized learning is an efficient emerging paradigm for boosting the computing capability of multiple bounded computing agents. In the big data era, performing inference within the distributed and federated learning (DL and FL) frameworks, the central server needs to process a large amount of data while relying on various agents to perform multiple distributed training tasks. Considering the decentralized computing topology, privacy has become a first-class concern. Moreover, assuming limited information processing capability for the agents calls for a sophisticated \textit{privacy-preserving decentralization} that ensures efficient computation. Towards this end, we study the \textit{privacy-aware server to multi-agent assignment} problem subject to information processing constraints associated with each agent, while maintaining the privacy and assuring learning informative messages received by agents about a global terminal through the distributed private federated learning (DPFL) approach. To find a decentralized scheme for a two-agent system, we formulate an optimization problem that balances privacy and accuracy, taking into account the quality of compression constraints associated with each agent. We propose an iterative converging algorithm by alternating over self-consistent equations. We also numerically evaluate the proposed solution to show the privacy-prediction trade-off and demonstrate the efficacy of the novel approach in ensuring privacy in DL and FL.