Expressive Power of Graph Neural Networks for (Mixed-Integer) Quadratic Programs

TL;DR

This paper investigates the theoretical capabilities of graph neural networks in solving quadratic programming problems, demonstrating their universal representation power for convex cases and identifying specific problem subclasses for mixed-integer cases.

Contribution

It provides the first theoretical analysis of GNNs' ability to represent quadratic programming solutions, including proofs of universality for convex QPs and problem subclass identification for mixed-integer QPs.

Findings

GNNs can universally represent convex QP properties.

GNNs are not universal for mixed-integer QPs.

A subclass of mixed-integer QPs can be reliably represented by GNNs.

Abstract

Quadratic programming (QP) is the most widely applied category of problems in nonlinear programming. Many applications require real-time/fast solutions, though not necessarily with high precision. Existing methods either involve matrix decomposition or use the preconditioned conjugate gradient method. For relatively large instances, these methods cannot achieve the real-time requirement unless there is an effective preconditioner. Recently, graph neural networks (GNNs) opened new possibilities for QP. Some promising empirical studies of applying GNNs for QP tasks show that GNNs can capture key characteristics of an optimization instance and provide adaptive guidance accordingly to crucial configurations during the solving process, or directly provide an approximate solution. However, the theoretical understanding of GNNs in this context remains limited. Specifically, it is unclear what GNNs can and cannot achieve for QP tasks in theory. This work addresses this gap in the context of linearly constrained QP tasks. In the continuous setting, we prove that message-passing GNNs can universally represent fundamental properties of convex quadratic programs, including feasibility, optimal objective values, and optimal solutions. In the more challenging mixed-integer setting, while GNNs are not universal approximators, we identify a subclass of QP problems that GNNs can reliably represent.