Top-k data selection via distributed sample quantile inference

TL;DR

This paper introduces a distributed stochastic approximation algorithm for top-$k$ data selection in noisy networked environments, effectively solving the sample quantile inference problem with proven convergence.

Contribution

It presents a novel two-time-scale stochastic approximation method for distributed sample quantile inference, with rigorous convergence guarantees and empirical efficiency.

Findings

Algorithm converges almost surely to the optimal solution.

Handles noise effectively in distributed settings.

Achieves accurate top-$k$ selection within few iterations.

Abstract

We consider the problem of determining the top-$k$ largest measurements from a dataset distributed among a network of $n$ agents with noisy communication links. We show that this scenario can be cast as a distributed convex optimization problem called sample quantile inference, which we solve using a two-time-scale stochastic approximation algorithm. Herein, we prove the algorithm's convergence in the almost sure sense to an optimal solution. Moreover, our algorithm handles noise and empirically converges to the correct answer within a small number of iterations.