Derivative of a Conic Problem with a Unique Solution

TL;DR

This paper establishes conditions under which the solution map of a conic optimization problem with a unique solution is differentiable, providing a derivative operator useful for applications like machine learning and control.

Contribution

It generalizes previous results by deriving the derivative of the solution map for any convex conic problem under regularity conditions, with a practical Python implementation.

Findings

Solution map is differentiable under regularity conditions.

Derived an abstract linear operator for the derivative.

Applicable to practical problems with cone constraints.

Abstract

We view a conic optimization problem that has a unique solution as a map from its data to its solution. If sufficient regularity conditions hold at a solution point, namely that the implicit function theorem applies to the normalized residual function of [Busseti et al., 2018], the problem solution map is differentiable. We obtain the derivative, in the form of an abstract linear operator. This applies to any convex optimization problem in conic form, while a previous result [Amos et al., 2016] studied strictly convex quadratic programs. Such differentiable problems can be used, for example, in machine learning, control, and related areas, as a layer in an end-to-end learning and control procedure, for backpropagation. We accompany this note with a lightweight Python implementation which can handle problems with the cone constraints commonly used in practice.