Objective-Function Free Multi-Objective Optimization: Rate of Convergence and Performance of an Adagrad-like algorithm

TL;DR

This paper introduces a novel Adagrad-like algorithm for multi-objective optimization that avoids dominance-based acceptance criteria, uses adaptive stepsizes without line searches, and demonstrates favorable convergence and robustness in experiments.

Contribution

It presents a new function-free, convergence-rate analyzed algorithm for multi-objective optimization that does not rely on dominance properties or line searches.

Findings

Convergence rate of O(1/√k) for the proposed method.

Effective performance on various unconstrained multi-objective problems.

Robustness under noisy multi-objective settings.

Abstract

We propose an Adagrad-like algorithm for multi-objective unconstrained optimization that relies on the computation of a common descent direction only. Unlike classical local algorithms for multi-objective optimization, our approach does not rely on the dominance property to accept new iterates, which allows for a flexible and function-free optimization framework. New points are obtained using an adaptive stepsize that does not require neither knowledge of Lipschitz constants nor the use of line search procedures. The rate of convergence is analyzed and is shown to be $\mathcal{O}(1 / \sqrt{ k+1})$ with respect to the norm of the common descent direction. The method is extensively validated on a broad class of unconstrained multi-objective problems and simple multi-task learning instances, and compared against a first-order line search algorithm. Additionally, we present a preliminary study of the behavior under noisy multi-objective settings, highlighting the robustness of the method.