Capacity of Hierarchical Secure Coded Gradient Aggregation with Straggling Communication Links

TL;DR

This paper introduces a new secure hierarchical coded gradient aggregation scheme for distributed learning systems with straggling links, ensuring privacy and optimality in a user-helper-master network.

Contribution

It formulates a novel hierarchical secure gradient aggregation problem and provides an optimal coding scheme with a matching converse bound for the first time.

Findings

Achieves secure gradient aggregation with straggling links.

Provides an optimal coding scheme with proven bounds.

Ensures privacy against colluding helpers and users.

Abstract

The growing privacy concerns in distributed learning have led to the widespread adoption of secure aggregation techniques in distributed machine learning systems, such as federated learning. Motivated by a coded gradient aggregation problem in a user-helper-master hierarchical network setting with straggling communication links, we formulate a new secure hierarchical coded gradient aggregation problem. In our setting, \( K \) users communicate with the master through an intermediate layer of \( N \) helpers, who can communicate with each other. With a resiliency threshold of \( N_r \) for straggling communication links, and at most \( T \) colluding helpers and any number of colluding users, the master aims to recover the sum of all users' gradients while remaining unaware of any individual gradient that exceeds the expected sum. In addition, helpers cannot infer more about users' gradients than what is already known by the colluding users. We propose an achievable scheme where users' upload messages are based on a globally known Vandermonde matrix, and helper communication is facilitated using an extended Vandermonde matrix with special structural properties. A matching converse bound is also derived, establishing the optimal result for this hierarchical coded gradient aggregation problem.