Supervised Learning Under Distributed Features

TL;DR

This paper introduces a distributed learning method for large-scale, high-dimensional data where features are distributed across networked agents, ensuring convergence to the global optimum efficiently.

Contribution

It proposes a novel distributed primal algorithm combining diffusion, pipeline, and variance reduction techniques with guaranteed linear convergence.

Findings

Converges to the global minimizer with linear rate

Effective in large datasets with distributed features

Validated through simulation results

Abstract

This work studies the problem of learning under both large datasets and large-dimensional feature space scenarios. The feature information is assumed to be spread across agents in a network, where each agent observes some of the features. Through local cooperation, the agents are supposed to interact with each other to solve an inference problem and converge towards the global minimizer of an empirical risk. We study this problem exclusively in the primal domain, and propose new and effective distributed solutions with guaranteed convergence to the minimizer with linear rate under strong convexity. This is achieved by combining a dynamic diffusion construction, a pipeline strategy, and variance-reduced techniques. Simulation results illustrate the conclusions.