Decentralized Federated Learning with Distributed Aggregation Weight Optimization

TL;DR

This paper proposes a fully distributed algorithm for optimizing aggregation weights in decentralized federated learning, enhancing learning efficiency and accuracy without relying on a central server.

Contribution

It introduces a distributed weight optimization method that aligns with decentralized DFL, including convergence analysis and a subgradient-based solution approach.

Findings

The proposed algorithm improves convergence speed in DFL.

Distributed weight optimization enhances model accuracy.

Numerical results confirm the algorithm's superiority in practical scenarios.

Abstract

Decentralized federated learning (DFL) is an emerging paradigm to enable edge devices collaboratively training a learning model using a device-to-device (D2D) communication manner without the coordination of a parameter server (PS). Aggregation weights, also known as mixing weights, are crucial in DFL process, and impact the learning efficiency and accuracy. Conventional design relies on a so-called central entity to collect all local information and conduct system optimization to obtain appropriate weights. In this paper, we develop a distributed aggregation weight optimization algorithm to align with the decentralized nature of DFL. We analyze convergence by quantitatively capturing the impact of the aggregation weights over decentralized communication networks. Based on the analysis, we then formulate a learning performance optimization problem by designing the aggregation weights to minimize the derived convergence bound. The optimization problem is further transformed as an eigenvalue optimization problem and solved by our proposed subgradient-based algorithm in a distributed fashion. In our algorithm, edge devices only need local information to obtain the optimal aggregation weights through local (D2D) communications, just like the learning itself. Therefore, the optimization, communication, and learning process can be all conducted in a distributed fashion, which leads to a genuinely distributed DFL system. Numerical results demonstrate the superiority of the proposed algorithm in practical DFL deployment.