SMOP: Stochastic trust region method for multi-objective problems

TL;DR

This paper introduces SMOP, a stochastic trust region algorithm for multi-objective optimization that handles noisy functions using probabilistic models, with proven convergence and demonstrated effectiveness in machine learning tasks.

Contribution

The paper develops a novel probabilistic trust region method for multi-objective problems, proving convergence and showing superior performance on complex, noisy, and irregular problems.

Findings

Proven almost sure convergence to Pareto critical points.

Outperforms existing stochastic multi-objective solvers in numerical tests.

Effectively trains non-discriminatory logistic regression models.

Abstract

The problem we consider is a multi-objective optimization problem, in which the goal is to find an optimal value of a vector function representing various criteria. The aim of this work is to develop an algorithm which utilizes the trust region framework with probabilistic model functions, able to cope with noisy problems, using inaccurate functions and gradients. The key novelty is approximation of each function in the multiobjective problem with probabilistically fully linear model which yields the composite model defined by max operator as a satisfactory approximation for the nonsmooth scalarized objective function. We prove the almost sure convergence of the proposed algorithm to a Pareto critical point. Numerical results demonstrate effectiveness of the probabilistic trust region by comparing it to competitive stochastic multi-objective solvers. The application in supervised machine learning is showcased by training non discriminatory Logistic Regression models on different size data groups. Additionally, we use several test examples with irregularly shaped fronts to exhibit the efficiency of the algorithm