Differentiating Through a Quadratic Cone Program

TL;DR

This paper develops methods to differentiate the solution map of quadratic cone programs, enabling efficient gradient computations and integration into neural networks, with an open-source GPU-compatible implementation.

Contribution

It generalizes differentiation techniques from linear to quadratic cone programs and provides an efficient, GPU-compatible implementation for practical use.

Findings

Differentiability of the solution map is established using the implicit function theorem.

The paper introduces exttt{diffqcp}, an open-source tool for derivative evaluation.

GPU implementation outperforms CPU for large quadratic cone programs.

Abstract

Quadratic cone programs are rapidly becoming the standard canonical form for convex optimization problems. In this paper we address the question of differentiating the solution map for such problems, generalizing previous work for linear cone programs. We follow a similar path, using the implicit function theorem applied to the optimality conditions for a homogenous primal-dual embedding. Along with our proof of differentiability, we present methods for efficiently evaluating the derivative operator and its adjoint at a vector. Additionally, we present an open-source implementation of these methods, named \texttt{diffqcp}, that can execute on CPUs and GPUs. GPU-compatibility is already of consequence as it enables convex optimization solvers to be integrated into neural networks with reduced data movement, but we go a step further demonstrating that \texttt{diffqcp}'s performance on GPUs surpasses the performance of its CPU-based counterpart for larger quadratic cone programs.