Iterative Implicit Gradients for Nonconvex Optimization with Variational Inequality Constraints

TL;DR

This paper introduces an iterative implicit gradient method for nonconvex constrained optimization, applicable to various machine learning tasks, with theoretical convergence guarantees and efficient large-scale computation strategies.

Contribution

It extends iterative differentiation to constrained bilevel problems, providing convergence analysis, error bounds, and scalable algorithms for complex machine learning applications.

Findings

Proposed an efficient implicit gradient computation strategy.

Established convergence and rate guarantees for the algorithm.

Applicable to diverse machine learning problems like meta-learning and reinforcement learning.

Abstract

We propose an optimization proxy in terms of iterative implicit gradient methods for solving constrained optimization problems with nonconvex loss functions. This framework can be applied to a broad range of machine learning settings, including meta-learning, hyperparameter optimization, large-scale complicated constrained optimization, and reinforcement learning. The proposed algorithm builds upon the iterative differentiation (ITD) approach. We extend existing convergence and rate analyses from the bilevel optimization literature to a constrained bilevel setting, motivated by learning under explicit constraints. Since solving bilevel problems using first-order methods requires evaluating the gradient of the inner-level optimal solution with respect to the outer variable (the implicit gradient), we develop an efficient computation strategy suitable for large-scale structures. Furthermore, we establish error bounds relative to the true gradients and provide non-asymptotic convergence rate guarantees.