On Constraint Qualifications for MPECs with Applications to Bilevel Hyperparameter Optimization for Machine Learning

TL;DR

This paper analyzes constraint qualifications for MPECs, focusing on bilevel hyperparameter optimization in machine learning, and provides a complete characterization of MPEC-LICQ for support vector machines.

Contribution

It clarifies relationships among classical MPEC constraint qualifications and characterizes MPEC-LICQ in the context of BHO for SVMs.

Findings

Complete characterization of MPEC-LICQ for BHO in SVMs.

Analysis of when constraint qualifications hold or fail in this context.

Clarification of relationships among classical MPEC constraint qualifications.

Abstract

Constraint qualifications for a Mathematical Program with Equilibrium Constraints (MPEC) are essential for analyzing stationarity properties and establishing convergence results. In this paper, we explore several classical MPEC constraint qualifications and clarify the relationships among them. We subsequently examine the behavior of these constraint qualifications in the context of a specific MPEC derived from bilevel hyperparameter optimization (BHO) for L1-loss support vector classification. In particular, for such an MPEC, we provide a complete characterization of the well-known MPEC linear independence constraint qualification (MPEC-LICQ), therefore, establishing conditions under which it holds or fails for our BHO for support vector machines.