Voronoi-grid-based Pareto Front Learning and Its Application to Collaborative Federated Learning

TL;DR

This paper introduces a Voronoi-grid-based Pareto front learning framework using genetic algorithms to improve coverage and efficiency in multi-objective federated learning tasks.

Contribution

It proposes a novel PHN-HVVS framework that decomposes the design space into Voronoi grids and employs genetic algorithms for better Pareto front approximation.

Findings

Outperforms baseline methods in Pareto front generation

Enhances coverage of the Pareto front, especially in high-dimensional spaces

Advances federated learning methodologies with multi-objective optimization

Abstract

Multi-objective optimization (MOO) exists extensively in machine learning, and aims to find a set of Pareto-optimal solutions, called the Pareto front, e.g., it is fundamental for multiple avenues of research in federated learning (FL). Pareto-Front Learning (PFL) is a powerful method implemented using Hypernetworks (PHNs) to approximate the Pareto front. This method enables the acquisition of a mapping function from a given preference vector to the solutions on the Pareto front. However, most existing PFL approaches still face two challenges: (a) sampling rays in high-dimensional spaces; (b) failing to cover the entire Pareto Front which has a convex shape. Here, we introduce a novel PFL framework, called as PHN-HVVS, which decomposes the design space into Voronoi grids and deploys a genetic algorithm (GA) for Voronoi grid partitioning within high-dimensional space. We put forward a new loss function, which effectively contributes to more extensive coverage of the resultant Pareto front and maximizes the HV Indicator. Experimental results on multiple MOO machine learning tasks demonstrate that PHN-HVVS outperforms the baselines significantly in generating Pareto front. Also, we illustrate that PHN-HVVS advances the methodologies of several recent problems in the FL field. The code is available at https://github.com/buptcmm/phnhvvs}{https://github.com/buptcmm/phnhvvs.