Approximation Methods for Bilevel Programming

TL;DR

This paper introduces approximation algorithms for bilevel programming problems with strongly convex inner objectives, providing convergence analysis and an accelerated variant, including stochastic settings with noisy information.

Contribution

It presents the first stochastic approximation algorithms with proven iteration and sample complexity for bilevel programming problems.

Findings

Finite-time convergence of the proposed algorithms.

Accelerated method improves convergence rate under convexity.

Extension to stochastic setting with noisy data.

Abstract

In this paper, we study a class of bilevel programming problem where the inner objective function is strongly convex. More specifically, under some mile assumptions on the partial derivatives of both inner and outer objective functions, we present an approximation algorithm for solving this class of problem and provide its finite-time convergence analysis under different convexity assumption on the outer objective function. We also present an accelerated variant of this method which improves the rate of convergence under convexity assumption. Furthermore, we generalize our results under stochastic setting where only noisy information of both objective functions is available. To the best of our knowledge, this is the first time that such (stochastic) approximation algorithms with established iteration complexity (sample complexity) are provided for bilevel programming.