Permutation Invariant Representations with Applications to Graph Deep Learning

TL;DR

This paper introduces two permutation-invariant Euclidean embeddings for graph data, enabling effective deep learning models that are robust to node relabeling, with theoretical guarantees and practical experiments on chemical and protein datasets.

Contribution

It proposes novel permutation-invariant embeddings based on sorting and polynomial algebras, with low complexity and strong theoretical properties for graph deep learning.

Findings

The sorting-based embedding is globally bi-Lipschitz with low-dimensional target space.

An almost everywhere injective embedding with minimal redundancy is achievable.

Numerical experiments demonstrate effectiveness on chemical and protein datasets.

Abstract

This paper presents primarily two Euclidean embeddings of the quotient space generated by matrices that are identified modulo arbitrary row permutations. The original application is in deep learning on graphs where the learning task is invariant to node relabeling. Two embedding schemes are introduced, one based on sorting and the other based on algebras of multivariate polynomials. While both embeddings exhibit a computational complexity exponential in problem size, the sorting based embedding is globally bi-Lipschitz and admits a low dimensional target space. Additionally, an almost everywhere injective scheme can be implemented with minimal redundancy and low computational cost. In turn, this proves that almost any classifier can be implemented with an arbitrary small loss of performance. Numerical experiments are carried out on two data sets, a chemical compound data set (QM9) and a proteins data set (PROTEINS).