Time-varying Mixing Matrix Design for Energy-efficient Decentralized Federated Learning

TL;DR

This paper proposes a novel design for time-varying mixing matrices in decentralized federated learning to minimize maximum per-node energy consumption, balancing energy efficiency and convergence speed in wireless networks.

Contribution

It introduces a theoretically-justified, multi-phase framework for designing time-varying mixing matrices that optimize energy use while ensuring convergence in DFL.

Findings

Validated the approach with real data showing energy savings.

Achieved a balance between sparse and dense mixing matrices.

Demonstrated improved energy efficiency without sacrificing convergence speed.

Abstract

We consider the design of mixing matrices to minimize the operation cost for decentralized federated learning (DFL) in wireless networks, with focus on minimizing the maximum per-node energy consumption. As a critical hyperparameter for DFL, the mixing matrix controls both the convergence rate and the needs of agent-to-agent communications, and has thus been studied extensively. However, existing designs mostly focused on minimizing the communication time, leaving open the minimization of per-node energy consumption that is critical for energy-constrained devices. This work addresses this gap through a theoretically-justified solution for mixing matrix design that aims at minimizing the maximum per-node energy consumption until convergence, while taking into account the broadcast nature of wireless communications. Based on a novel convergence theorem that allows arbitrarily time-varying mixing matrices, we propose a multi-phase design framework that activates time-varying communication topologies under optimized budgets to trade off the per-iteration energy consumption and the convergence rate while balancing the energy consumption across nodes. Our evaluations based on real data have validated the efficacy of the proposed solution in combining the low energy consumption of sparse mixing matrices and the fast convergence of dense mixing matrices.