Solving smooth min-min and min-max problems by mixed oracle algorithms

TL;DR

This paper introduces a framework for solving smooth min-min and min-max problems using mixed oracle algorithms, combining gradient and zeroth-order evaluations to achieve non-asymptotic complexity bounds.

Contribution

It develops a novel approach that integrates inexact oracles and accelerated methods for efficient solution of smooth min-min and min-max problems.

Findings

Provides non-asymptotic complexity bounds for both problem types.

Estimates the number of gradient and zeroth-order oracle calls needed.

Demonstrates effectiveness of the mixed oracle approach.

Abstract

In this paper, we consider two types of problems that have some similarity in their structure, namely, min-min problems and min-max saddle-point problems. Our approach is based on considering the outer minimization problem as a minimization problem with inexact oracle. This inexact oracle is calculated via inexact solution of the inner problem, which is either minimization or a maximization problem. Our main assumptions are that the problem is smooth and the available oracle is mixed: it is only possible to evaluate the gradient w.r.t. the outer block of variables which corresponds to the outer minimization problem, whereas for the inner problem only zeroth-order oracle is available. To solve the inner problem we use accelerated gradient-free method with zeroth-order oracle. To solve the outer problem we use either inexact variant of Vaydya's cutting-plane method or a variant of accelerated gradient method. As a result, we propose a framework that leads to non-asymptotic complexity bounds for both min-min and min-max problems. Moreover, we estimate separately the number of first- and zeroth-order oracle calls which are sufficient to reach any desired accuracy.