Quantitative Bounds for Sorting-Based Permutation-Invariant Embeddings

TL;DR

This paper investigates permutation-invariant embeddings based on sorting projections, improving bounds on the embedding dimension for injectivity and analyzing the bi-Lipschitz distortion, with implications for graph deep learning.

Contribution

It provides new bounds on the minimal embedding dimension for injectivity and constructs projection matrices with quadratic bi-Lipschitz distortion dependence on the number of points.

Findings

Improved upper bounds for embedding dimension D for injectivity.

Constructed projection matrices with quadratic bi-Lipschitz distortion dependence on n.

Established lower bounds on the minimal injectivity dimension.

Abstract

We study permutation-invariant embeddings of $d$-dimensional point sets, which are defined by sorting $D$ independent one-dimensional projections of the input. Such embeddings arise in graph deep learning where outputs should be invariant to permutations of graph nodes. Previous work showed that for large enough $D$ and projections in general position, this mapping is injective, and moreover satisfies a bi-Lipschitz condition. However, two gaps remain: firstly, the optimal size $D$ required for injectivity is not yet known, and secondly, no estimates of the bi-Lipschitz constants of the mapping are known. In this paper, we make substantial progress in addressing both of these gaps. Regarding the first gap, we improve upon the best known upper bounds for the embedding dimension $D$ necessary for injectivity, and also provide a lower bound on the minimal injectivity dimension. Regarding the second gap, we construct matrices of projection vectors, so that the bi-Lipschitz distortion of the mapping depends quadratically on the number of points $n$, and is completely independent of the dimension $d$. We also show that for any choice of projection vectors, the distortion of the mapping will never be better than a bound proportional to the square root of $n$. Finally, we show that similar guarantees can be provided even when linear projections are applied to the mapping to reduce its dimension.