Federated Learning Under Restricted User Availability

TL;DR

This paper introduces a risk-aware federated learning framework that effectively handles limited user participation by employing a CVaR-based objective, improving performance in environments with restricted or infrequent user availability.

Contribution

It proposes a novel formulation of federated learning that accounts for restricted user participation using a CVaR-based risk-aware objective and an efficient, RAM-agnostic training algorithm.

Findings

Significantly improved performance over standard FL in various setups.

Effective handling of limited user participation through risk-aware optimization.

Maintains computational complexity similar to FedAvg.

Abstract

Federated Learning (FL) is a decentralized machine learning framework that enables collaborative model training while respecting data privacy. In various applications, non-uniform availability or participation of users is unavoidable due to an adverse or stochastic environment, the latter often being uncontrollable during learning. Here, we posit a generic user selection mechanism implementing a possibly randomized, stationary selection policy, suggestively termed as a Random Access Model (RAM). We propose a new formulation of the FL problem which effectively captures and mitigates limited participation of data originating from infrequent, or restricted users, at the presence of a RAM. By employing the Conditional Value-at-Risk (CVaR) over the (unknown) RAM distribution, we extend the expected loss FL objective to a risk-aware objective, enabling the design of an efficient training algorithm that is completely oblivious to the RAM, and with essentially identical complexity as FedAvg. Our experiments on synthetic and benchmark datasets show that the proposed approach achieves significantly improved performance as compared with standard FL, under a variety of setups.