Techniques for Gradient Based Bilevel Optimization with Nonsmooth Lower Level Problems

TL;DR

This paper introduces gradient-based techniques for bilevel optimization problems with non-smooth lower levels, using iterative algorithms and differentiable proximal mappings to handle non-smoothness and non-uniqueness.

Contribution

It presents a novel approach to approximate solutions of non-smooth bilevel problems by replacing minimizers with differentiable iterative algorithms.

Findings

Effective approximation of non-smooth bilevel problems

Differentiable update mappings enable gradient-based optimization

Handles non-uniqueness in lower level solutions

Abstract

We propose techniques for approximating bilevel optimization problems with non-smooth lower level problems that can have a non-unique solution. To this end, we substitute the expression of a minimizer of the lower level minimization problem with an iterative algorithm that is guaranteed to converge to a minimizer of the problem. Using suitable non-linear proximal distance functions, the update mappings of such an iterative algorithm can be differentiable, notwithstanding the fact that the minimization problem is non-smooth.