Federated Multi-Objective Learning with Controlled Pareto Frontiers

TL;DR

This paper introduces CR-FMOL, a federated multi-objective learning framework that enforces client fairness by ensuring Pareto optimality for each client through a novel preference-cone constraint, improving fairness in non-IID settings.

Contribution

CR-FMOL is the first federated MOO framework to explicitly enforce client-wise Pareto optimality using a preference-cone constraint.

Findings

Enhances client fairness in federated learning.

Slightly lower early-stage performance compared to FedAvg.

Expected to achieve comparable accuracy with more training rounds.

Abstract

Federated learning (FL) is a widely adopted paradigm for privacy-preserving model training, but FedAvg optimise for the majority while under-serving minority clients. Existing methods such as federated multi-objective learning (FMOL) attempts to import multi-objective optimisation (MOO) into FL. However, it merely delivers task-wise Pareto-stationary points, leaving client fairness to chance. In this paper, we introduce Conically-Regularised FMOL (CR-FMOL), the first federated MOO framework that enforces client-wise Pareto optimality through a novel preference-cone constraint. After local federated multi-gradient descent averaging (FMGDA) / federated stochastic multi-gradient descent averaging (FSMGDA) steps, each client transmits its aggregated task-loss vector as an implicit preference; the server then solves a cone-constrained Pareto-MTL sub-problem centred at the uniform vector, producing a descent direction that is Pareto-stationary for every client within its cone. Experiments on non-IID benchmarks show that CR-FMOL enhances client fairness, and although the early-stage performance is slightly inferior to FedAvg, it is expected to achieve comparable accuracy given sufficient training rounds.