Broad Spectrum Structure Discovery in Large-Scale Higher-Order Networks

TL;DR

This paper presents a probabilistic framework for efficiently discovering diverse mesoscale structures in large-scale hypergraphs, enabling better understanding and prediction of complex systems with higher-order interactions.

Contribution

It introduces a novel latent hypergraph model that captures rich structural patterns using low-rank representations, improving interpretability and scalability.

Findings

Enhanced link prediction accuracy over existing methods

Discovery of interpretable structures in real-world networks

Scalable analysis of large hypergraph datasets

Abstract

Complex systems are often driven by higher-order interactions among multiple units, naturally represented as hypergraphs. Understanding dependency structures within these hypergraphs is crucial for understanding and predicting the behavior of complex systems but is made challenging by their combinatorial complexity and computational demands. In this paper, we introduce a class of probabilistic models that efficiently represents and discovers a broad spectrum of mesoscale structure in large-scale hypergraphs. The key insight enabling this approach is to treat classes of similar units as themselves nodes in a latent hypergraph. By modeling observed node interactions through latent interactions among classes using low-rank representations, our approach tractably captures rich structural patterns while ensuring model identifiability. This allows for direct interpretation of distinct node- and class-level structures. Empirically, our model improves link prediction over state-of-the-art methods and discovers interpretable structures in diverse real-world systems, including pharmacological and social networks, advancing the ability to incorporate large-scale higher-order data into the scientific process.