Sparse Uncertainty-Informed Sampling from Federated Streaming Data

TL;DR

This paper introduces a robust and efficient sampling method for federated streaming data that improves model training by selecting relevant observations without memory buffers, especially under resource constraints.

Contribution

It proposes a novel, uncertainty-informed sampling approach tailored for federated streaming data, enhancing robustness and efficiency without relying on buffering strategies.

Findings

Improved training batch diversity in federated learning.

Enhanced numerical robustness over existing methods.

Effective in large-scale, resource-limited environments.

Abstract

We present a numerically robust, computationally efficient approach for non-I.I.D. data stream sampling in federated client systems, where resources are limited and labeled data for local model adaptation is sparse and expensive. The proposed method identifies relevant stream observations to optimize the underlying client model, given a local labeling budget, and performs instantaneous labeling decisions without relying on any memory buffering strategies. Our experiments show enhanced training batch diversity and an improved numerical robustness of the proposal compared to existing strategies over large-scale data streams, making our approach an effective and convenient solution in FL environments.