A Multi-Objective Optimization framework for Decentralized Learning with coordination constraints

TL;DR

This paper presents a flexible multi-objective framework for decentralized learning that explicitly models coordinator and agent objectives, offering a new way to balance personalization, fairness, and coordination.

Contribution

It introduces a generalized multi-objective formulation with scalarization strategies and provides a decentralized algorithm with convergence guarantees.

Findings

Proposed a Pareto optimal solution under convexity assumptions.

Developed a decentralized optimization algorithm with convergence guarantees.

Demonstrated empirical effectiveness through simulations.

Abstract

This article introduces a generalized framework for Decentralized Learning formulated as a Multi-Objective Optimization problem, in which both distributed agents and a central coordinator contribute independent, potentially conflicting objectives over a shared model parameter space. Unlike traditional approaches that merge local losses under a common goal, our formulation explicitly incorporates coordinator-side criteria, enabling more flexible and structured training dynamics. To navigate the resulting trade-offs, we explore scalarization strategies, particularly weighted sums, to construct tractable surrogate problems. These yield solutions that are provably Pareto optimal under standard convexity and smoothness assumptions, while embedding global preferences directly into local updates. We propose a decentralized optimization algorithm with convergence guarantees, and demonstrate its empirical performance through simulations, highlighting the impact of the coordinator's influence on local agent behavior. The proposed approach offers a principled and customizable strategy for balancing personalization, fairness, and coordination in decentralized learning systems.