Deep Message Passing on Sets

TL;DR

Deep Message Passing on Sets (DMPS) introduces a novel relational learning method that enhances set data processing by integrating graph learning, diffusion dynamics, and residual connections, demonstrating superior performance on various datasets.

Contribution

DMPS develops new relational learning blocks for sets, bridging message passing, diffusion models, and residual networks, with demonstrated interpretability and improved results.

Findings

Learned true underlying relational structures experimentally.

Achieved competitive or superior results on synthetic and real-world datasets.

Demonstrated interpretability of the relational structures learned.

Abstract

Modern methods for learning over graph input data have shown the fruitfulness of accounting for relationships among elements in a collection. However, most methods that learn over set input data use only rudimentary approaches to exploit intra-collection relationships. In this work we introduce Deep Message Passing on Sets (DMPS), a novel method that incorporates relational learning for sets. DMPS not only connects learning on graphs with learning on sets via deep kernel learning, but it also bridges message passing on sets and traditional diffusion dynamics commonly used in denoising models. Based on these connections, we develop two new blocks for relational learning on sets: the set-denoising block and the set-residual block. The former is motivated by the connection between message passing on general graphs and diffusion-based denoising models, whereas the latter is inspired by the well-known residual network. In addition to demonstrating the interpretability of our model by learning the true underlying relational structure experimentally, we also show the effectiveness of our approach on both synthetic and real-world datasets by achieving results that are competitive with or outperform the state-of-the-art.