Representation Learning with Multisets

TL;DR

This paper introduces a measure-theoretic approach to learning permutation-invariant multiset representations, demonstrating improved prediction of containment and set operations, and learning meaningful representations.

Contribution

It formalizes multiset representations using measure theory and proposes a novel training task for better capturing containment relations.

Findings

Outperforms DeepSets in predicting symmetric difference and intersection sizes

Learns meaningful, flexible multiset representations

Effective at modeling containment relations

Abstract

We study the problem of learning permutation invariant representations that can capture "flexible" notions of containment. We formalize this problem via a measure theoretic definition of multisets, and obtain a theoretically-motivated learning model. We propose training this model on a novel task: predicting the size of the symmetric difference (or intersection) between pairs of multisets. We demonstrate that our model not only performs very well on predicting containment relations (and more effectively predicts the sizes of symmetric differences and intersections than DeepSets-based approaches with unconstrained object representations), but that it also learns meaningful representations.