Optimizing over trained GNNs via symmetry breaking

TL;DR

This paper develops symmetry-breaking constraints and optimization formulations for trained GNNs, enabling effective optimization over graph structures, with applications demonstrated in molecular design.

Contribution

It introduces novel symmetry-breaking constraints and mixed-integer optimization formulations for GNNs, addressing graph isomorphism issues in model optimization.

Findings

Symmetry-breaking constraints improve optimization efficiency.

Optimization over GNNs with variable graphs is feasible with proposed methods.

Application in molecular design demonstrates practical effectiveness.

Abstract

Optimization over trained machine learning models has applications including: verification, minimizing neural acquisition functions, and integrating a trained surrogate into a larger decision-making problem. This paper formulates and solves optimization problems constrained by trained graph neural networks (GNNs). To circumvent the symmetry issue caused by graph isomorphism, we propose two types of symmetry-breaking constraints: one indexing a node 0 and one indexing the remaining nodes by lexicographically ordering their neighbor sets. To guarantee that adding these constraints will not remove all symmetric solutions, we construct a graph indexing algorithm and prove that the resulting graph indexing satisfies the proposed symmetry-breaking constraints. For the classical GNN architectures considered in this paper, optimizing over a GNN with a fixed graph is equivalent to optimizing over a dense neural network. Thus, we study the case where the input graph is not fixed, implying that each edge is a decision variable, and develop two mixed-integer optimization formulations. To test our symmetry-breaking strategies and optimization formulations, we consider an application in molecular design.