Pareto Actor-Critic for Communication and Computation Co-Optimization in Non-Cooperative Federated Learning Services

TL;DR

This paper introduces PAC-MCoFL, a game-theoretic MARL framework for non-cooperative federated learning that optimizes communication and computation resources, achieving Pareto-efficient equilibria with theoretical guarantees and superior simulation results.

Contribution

It proposes a novel Pareto Actor-Critic MARL framework with expectile regression and a ternary Cartesian decomposition for resource optimization in federated learning.

Findings

Achieves approximately 5.8% improvement in total reward

Achieves approximately 4.2% improvement in hypervolume indicator

Demonstrates effective balancing of individual and system performance

Abstract

Federated learning (FL) in multi-service provider (SP) ecosystems is fundamentally hampered by non-cooperative dynamics, where privacy constraints and competing interests preclude the centralized optimization of multi-SP communication and computation resources. In this paper, we introduce PAC-MCoFL, a game-theoretic multi-agent reinforcement learning (MARL) framework where SPs act as agents to jointly optimize client assignment, adaptive quantization, and resource allocation. Within the framework, we integrate Pareto Actor-Critic (PAC) principles with expectile regression, enabling agents to conjecture optimal joint policies to achieve Pareto-optimal equilibria while modeling heterogeneous risk profiles. To manage the high-dimensional action space, we devise a ternary Cartesian decomposition (TCAD) mechanism that facilitates fine-grained control. Further, we develop PAC-MCoFL-p, a scalable variant featuring a parameterized conjecture generator that substantially reduces computational complexity with a provably bounded error. Alongside theoretical convergence guarantees, our framework's superiority is validated through extensive simulations -- PAC-MCoFL achieves approximately 5.8% and 4.2% improvements in total reward and hypervolume indicator (HVI), respectively, over the latest MARL solutions. The results also demonstrate that our method can more effectively balance individual SP and system performance in scaled deployments and under diverse data heterogeneity.