Distributed Machine Learning with Strategic Network Design: A Game-Theoretic Perspective

TL;DR

This paper introduces a game-theoretic framework for distributed machine learning over networks, where nodes strategically choose learning parameters and network structures to optimize local and global performance.

Contribution

It develops a joint learning and network formation approach modeled as a potential game, with algorithms for convergence and network optimization, including extensions for streaming data.

Findings

The proposed game achieves better social welfare than fixed-network methods.

The algorithms converge to Nash equilibria in undirected networks.

Application to telemonitoring demonstrates practical effectiveness.

Abstract

This paper considers a game-theoretic framework for distributed machine learning problems over networks where the information acquisition at a node is modeled as a rational choice of a player. In the proposed game, players decide both the learning parameters and the network structure. The Nash equilibrium characterizes the tradeoff between the local performance and the global agreement of the learned classifiers. We first introduce a commutative approach which features a joint learning process that integrates the iterative learning at each node and the network formation. We show that our game is equivalent to a generalized potential game in the setting of undirected networks. We study the convergence of the proposed commutative algorithm, analyze the network structures determined by our game, and show the improvement of the social welfare in comparison with standard distributed learning over fixed networks. To adapt our framework to streaming data, we derive a distributed Kalman filter. A concurrent algorithm based on the online mirror descent algorithm is also introduced for solving for Nash equilibria in a holistic manner. In the case study, we use telemonitoring of Parkinson's disease to corroborate the results.