Distributed Perceptron under Bounded Staleness, Partial Participation, and Noisy Communication

TL;DR

This paper analyzes a distributed perceptron training method that accounts for delays, partial participation, and noisy communication, providing theoretical bounds on mistake rates under realistic system effects.

Contribution

It introduces a staleness-bucket aggregation rule with padding and derives mistake bounds considering delays, partial participation, and communication noise.

Findings

Mistake bounds depend on mean enforced staleness and noise energy.

Finite mistake budget leads to stabilization under mild conditions.

Delay impacts are isolated from communication noise in the bounds.

Abstract

We study a semi-asynchronous client-server perceptron trained via iterative parameter mixing (IPM-style averaging): clients run local perceptron updates and a server forms a global model by aggregating the updates that arrive in each communication round. The setting captures three system effects in federated and distributed deployments: (i) stale updates due to delayed model delivery and delayed application of client computations (two-sided version lag), (ii) partial participation (intermittent client availability), and (iii) imperfect communication on both downlink and uplink, modeled as effective zero-mean additive noise with bounded second moment. We introduce a server-side aggregation rule called staleness-bucket aggregation with padding that deterministically enforces a prescribed staleness profile over update ages without assuming any stochastic model for delays or participation. Under margin separability and bounded data radius, we prove a finite-horizon expected bound on the cumulative weighted number of perceptron mistakes over a given number of server rounds: the impact of delay appears only through the mean enforced staleness, whereas communication noise contributes an additional term that grows on the order of the square root of the horizon with the total noise energy. In the noiseless case, we show how a finite expected mistake budget yields an explicit finite-round stabilization bound under a mild fresh-participation condition.