Learning Convex Optimization Models

TL;DR

This paper introduces a heuristic method for learning parameters in convex optimization models, which encompass many classical models, by differentiating solutions with respect to parameters and demonstrates its application across three model classes.

Contribution

It proposes a novel heuristic for parameter learning in convex optimization models using differentiation of solutions, covering MAP, utility, and agent models.

Findings

Effective parameter learning demonstrated on three model classes.

Differentiation-based approach enables efficient training.

Applicable to a broad class of convex models.

Abstract

A convex optimization model predicts an output from an input by solving a convex optimization problem. The class of convex optimization models is large, and includes as special cases many well-known models like linear and logistic regression. We propose a heuristic for learning the parameters in a convex optimization model given a dataset of input-output pairs, using recently developed methods for differentiating the solution of a convex optimization problem with respect to its parameters. We describe three general classes of convex optimization models, maximum a posteriori (MAP) models, utility maximization models, and agent models, and present a numerical experiment for each.