Differentially-Private Multi-Tier Federated Learning: A Formal Analysis and Evaluation

TL;DR

This paper introduces M2FDP, a multi-tier federated learning framework with differential privacy, analyzing its convergence and proposing an adaptive algorithm to optimize privacy, performance, and resource efficiency in hierarchical networks.

Contribution

It develops M2FDP, a novel multi-tier DP federated learning method with hierarchical noise injection and convergence analysis, addressing privacy-performance trade-offs in complex networks.

Findings

M2FDP converges sublinearly under certain conditions.

Adaptive control improves energy, latency, and privacy trade-offs.

Numerical results outperform baseline methods across configurations.

Abstract

While federated learning (FL) eliminates the transmission of raw data over a network, it is still vulnerable to privacy breaches from the communicated model parameters. Differential privacy (DP) is often employed to address such issues. However, the impact of DP on FL in multi-tier networks -- where hierarchical aggregations couple noise injection decisions at different tiers, and trust models are heterogeneous across subnetworks -- is not well understood. To fill this gap, we develop \underline{M}ulti-Tier \underline{F}ederated Learning with \underline{M}ulti-Tier \underline{D}ifferential \underline{P}rivacy ({\tt M$^2$FDP}), a DP-enhanced FL methodology for jointly optimizing privacy and performance over such networks. One of the key principles of {\tt M$^2$FDP} is to adapt DP noise injection across the established edge/fog computing hierarchy (e.g., edge devices, intermediate nodes, and other tiers up to cloud servers) according to the trust models in different subnetworks. We conduct a comprehensive analysis of the convergence behavior of {\tt M$^2$FDP} under non-convex problem settings, revealing conditions on parameter tuning under which the training process converges sublinearly to a finite stationarity gap that depends on the network hierarchy, trust model, and target privacy level. We show how these relationships can be employed to develop an adaptive control algorithm for {\tt M$^2$FDP} that tunes properties of local model training to minimize energy, latency, and the stationarity gap while meeting desired convergence and privacy criterion. Subsequent numerical evaluations demonstrate that {\tt M$^2$FDP} obtains substantial improvements in these metrics over baselines for different privacy budgets and system configurations.