B\'ezier Flow: a Surface-wise Gradient Descent Method for Multi-objective Optimization

TL;DR

This paper introduces Bézier Flow, a novel surface-wise gradient descent method for multi-objective optimization, leveraging Bézier simplex models and PAC stability to improve generalization and stability of algorithms.

Contribution

It presents a new strategy to convert single-objective algorithms into multi-objective ones using Bézier simplexes and extends PAC stability to analyze their generalization performance.

Findings

Achieved lower generalization errors than existing methods.

Proved PAC stability leads to high-probability generalization bounds.

Demonstrated effectiveness through numerical experiments.

Abstract

In this paper, we propose a strategy to construct a multi-objective optimization algorithm from a single-objective optimization algorithm by using the B\'ezier simplex model. Also, we extend the stability of optimization algorithms in the sense of Probability Approximately Correct (PAC) learning and define the PAC stability. We prove that it leads to an upper bound on the generalization with high probability. Furthermore, we show that multi-objective optimization algorithms derived from a gradient descent-based single-objective optimization algorithm are PAC stable. We conducted numerical experiments and demonstrated that our method achieved lower generalization errors than the existing multi-objective optimization algorithm.