Variational Analysis of a Nonconvex and Nonsmooth Optimization Problem: An Introduction

TL;DR

This paper introduces variational analysis techniques for nonconvex, nonsmooth optimization, emphasizing practical tools and concepts like subgradients, duality, and second-order theory to advance algorithm development.

Contribution

It provides an accessible survey of variational analysis methods applied to broad classes of nonconvex, nonsmooth problems, including duality and second-order stability analysis.

Findings

Introduction of subgradients and optimality conditions

Derivation of dual problems and relaxations

Overview of second-order stability analysis

Abstract

Variational analysis provides the theoretical foundations and practical tools for constructing optimization algorithms without being restricted to smooth or convex problems. We survey the central concepts in the context of a concrete but broadly applicable problem class from composite optimization in finite dimensions. While prioritizing accessibility over mathematical details, we introduce subgradients of arbitrary functions and the resulting optimality conditions, describe approximations and the need for going beyond pointwise and uniform convergence, and summarize proximal methods. We derive dual problems from parametrization of the actual problem and the resulting relaxations. The paper ends with an introduction to second-order theory and its role in stability analysis of optimization problems.