Derivative-Free Bilevel Optimization with Inexact Lower-Level Solutions

TL;DR

This paper introduces a derivative-free bilevel optimization framework that handles complex constraints and inexact lower-level solutions, demonstrating improved performance with adaptive accuracy strategies in numerical experiments.

Contribution

It develops a novel derivative-free approach for bilevel problems with inexact lower-level solutions and complex constraints, with proven convergence properties.

Findings

Adaptive accuracy improves optimization results.

Convergence to stationary points is established.

Method outperforms fixed-precision approaches in experiments.

Abstract

In this work, we propose derivative-free framework for bilevel optimization. We consider both the upper and lower-level problems with bound constraints on the variables, as well as general nonlinear constraints, assuming that first-order information (in the upper-level) is not available or it is impractical to obtain. The lower-level problem is solved with an accuracy that is progressively refined throughout the optimization process. We first analyze the case in which the upper-level problem is subject only to bound constraints, establishing convergence to Clarke-Jahn stationary points when the refinement process is allowed to reach its maximum precision. When a limitation is imposed on this refinement process, we prove convergence to approximate stationary points using an extended notion of Goldstein stationarity. Finally, we extend the proposed framework to handle more complex constraints via an exact penalty function approach, proving convergence to stationary points under suitable assumptions. A comprehensive numerical study on 160 problems from the BOLIB collection shows that the adaptive accuracy strategy consistently yields better results than fixed-precision solves, with its benefits becoming more pronounced as the required lower-level accuracy becomes more stringent.