On the (Non) Injectivity of Piecewise Linear Janossy Pooling

TL;DR

This paper investigates the limitations of piecewise linear Janossy pooling functions, proving they cannot be injective for multisets with multiplicities, while simpler models suffice for injectivity in multisets without multiplicities.

Contribution

It provides a theoretical negative result on the injectivity of piecewise linear Janossy pooling functions and highlights simpler models' effectiveness for certain multiset types.

Findings

Piecewise linear Janossy pooling functions are not injective for multisets with multiplicities.

Simpler deep-sets models are sufficient for injectivity in multisets without multiplicities.

The study clarifies the limitations and capabilities of various multiset function models.

Abstract

Multiset functions, which are functions that map multisets to vectors, are a fundamental tool in the construction of neural networks for multisets and graphs. To guarantee that the vector representation of the multiset is faithful, it is often desirable to have multiset mappings that are both injective and bi-Lipschitz. Currently, there are several constructions of multiset functions achieving both these guarantees, leading to improved performance in some tasks but often also to higher compute time than standard constructions. Accordingly, it is natural to inquire whether simpler multiset functions achieving the same guarantees are available. In this paper, we make a large step towards giving a negative answer to this question. We consider the family of k-ary Janossy pooling, which includes many of the most popular multiset models, and prove that no piecewise linear Janossy pooling function can be injective. On the positive side, we show that when restricted to multisets without multiplicities, even simple deep-sets models suffice for injectivity and bi-Lipschitzness.