Distributed Learning with Adversarial Gradient Perturbations

TL;DR

This paper investigates the limits of distributed learning when clients provide adversarially perturbed gradients, establishing optimal bounds and algorithms for minimizing sub-optimality under such conditions.

Contribution

It introduces tight bounds on sub-optimality and develops algorithms with provable query complexity guarantees for learning with adversarial gradient perturbations.

Findings

Established tight feasibility thresholds for sub-optimality gap.

Designed algorithms achieving optimal sub-optimality with provable query complexity.

Analyzed the impact of adversarial gradient deviations on learning performance.

Abstract

Privacy concerns in distributed learning often lead clients to return intentionally altered gradient information. We consider the problem of learning convex and $L$-smooth functions under adversarial gradient perturbation, where a client's gradient reply to a server query can deviate arbitrarily from the true gradient subject to a distance bound. Our study focuses on two fundamental questions: (i) what is the smallest achievable sub-optimality gap (i.e., excess error in optimization) under such responses, and (ii) how many queries are sufficient to guarantee a given sub-optimality gap? We establish tight feasibility thresholds on the sub-optimality gap and provide algorithms that achieve these thresholds with provable query complexity guarantees.