Exploiting block triangular submatrices in KKT systems

TL;DR

This paper introduces a novel method for solving KKT systems by exploiting block triangular submatrices, leading to significant computational speedups in optimization problems involving neural network surrogates.

Contribution

The paper presents a new approach that uses Schur complement decomposition and block backsolve techniques to efficiently factorize KKT matrices, reducing fill-in and computational cost.

Findings

Achieves up to 15x speedup over existing methods

Effectively exploits block triangular structure in KKT matrices

Demonstrates advantages in neural network constrained optimization

Abstract

We propose a method for solving Karush-Kuhn-Tucker (KKT) systems that exploits block triangular submatrices by first using a Schur complement decomposition to isolate the block triangular submatrices then performing a block backsolve where only diagonal blocks of the block triangular form need to be factorized. We show that factorizing reducible symmetric-indefinite matrices with standard 1$\times$1 or 2$\times$2 pivots yields fill-in outside the diagonal blocks of the block triangular form, in contrast to our proposed method. While exploiting a block triangular submatrix has limited fill-in, unsymmetric matrix factorization methods do not reveal inertia, which is required by interior point methods for nonconvex optimization. We show that our target matrix has inertia that is known \textit{a priori}, letting us compute inertia of the KKT matrix by Sylvester's law. Finally, we demonstrate the computational advantage of this method on KKT systems from optimization problems with neural network surrogates in their constraints. Our method achieves up to 15$\times$ speedups over state-of-the-art symmetric indefinite matrix factorization methods MA57 and MA86 in a constant-hardware comparison.