Rep the Set: Neural Networks for Learning Set Representations

TL;DR

RepSet introduces a neural network architecture capable of processing set-structured data by computing set correspondences through network flow problems, enabling end-to-end learning for tasks like text and graph classification.

Contribution

The paper presents RepSet, a novel neural network design that effectively models set-structured data using network flow computations, a capability lacking in traditional models.

Findings

Achieves competitive or superior performance on classification tasks.

Handles variable-sized, unordered set inputs effectively.

Supports end-to-end gradient-based training.

Abstract

In several domains, data objects can be decomposed into sets of simpler objects. It is then natural to represent each object as the set of its components or parts. Many conventional machine learning algorithms are unable to process this kind of representations, since sets may vary in cardinality and elements lack a meaningful ordering. In this paper, we present a new neural network architecture, called RepSet, that can handle examples that are represented as sets of vectors. The proposed model computes the correspondences between an input set and some hidden sets by solving a series of network flow problems. This representation is then fed to a standard neural network architecture to produce the output. The architecture allows end-to-end gradient-based learning. We demonstrate RepSet on classification tasks, including text categorization, and graph classification, and we show that the proposed neural network achieves performance better or comparable to state-of-the-art algorithms.