ASMOP: Additional sampling stochastic trust region method for multi-objective problems

TL;DR

This paper introduces ASMOP, a stochastic trust region method with additional sampling for multi-objective optimization, demonstrating convergence and efficiency in machine learning tasks.

Contribution

It extends the trust region framework with additional sampling to handle multi-objective problems, providing convergence analysis and empirical validation.

Findings

Proves stochastic convergence for non-convex objectives.

Shows competitive performance on machine learning datasets.

Adapts sampling strategies to problem characteristics.

Abstract

We consider unconstrained multi-criteria optimization problems with finite sum objective functions. The proposed algorithm belongs to a non-monotone trust region framework where additional sampling approach is used to govern the sample size and the acceptance of a candidate point. Depending on the problem, the method can yield a mini-batch or an increasing sample size behavior. This work can be viewed as an extension of additional sampling trust region method for scalar finite sum function minimization presented in the literature, requiring nontrivial modifications both in construction and in convergence analysis of the algorithm. Under assumptions standard for this framework, we prove stochastic convergence for twice continuously-differentiable, but possibly non-convex objective functions. The experiments on machine learning binary classification datasets show the efficiency of the proposed scheme and its competitiveness with the relevant state-of-the-art methods in both convex and non-convex setup.