Inferring Higher-Order Couplings with Neural Networks

TL;DR

This paper introduces a novel method that uses Restricted Boltzmann Machines to efficiently infer higher-order interactions in complex systems, outperforming traditional models in accuracy and computational cost.

Contribution

The authors develop a systematic approach to extract higher-order couplings from RBMs by mapping them onto generalized Potts models, enabling efficient analysis of complex interactions.

Findings

Accurately recovers two- and three-body interactions in synthetic data.

Reconstructs protein contact maps with high fidelity.

Outperforms state-of-the-art inverse Potts models in real data.

Abstract

Maximum entropy methods, rooted in the inverse Ising/Potts problem from statistical physics, are widely used to model pairwise interactions in complex systems across disciplines such as bioinformatics and neuroscience. While successful, these approaches often fail to capture higher-order interactions that are critical for understanding collective behavior. In contrast, modern machine learning methods can model such interactions, but their interpretability often comes at a prohibitive computational cost. Restricted Boltzmann Machines (RBMs) provide a computationally efficient alternative by encoding statistical correlations through hidden units in a bipartite architecture. In this work, we introduce a method that maps RBMs onto generalized Potts models, enabling the systematic extraction of interactions up to arbitrary order. Leveraging large-$N$ approximations, made tractable by the RBM's structure, we extract effective many-body couplings with minimal computational effort. We further propose a robust framework for recovering higher-order interactions in more complex generative models, and introduce a simple gauge-fixing scheme for the effective Potts representation. Validation on synthetic data demonstrates accurate recovery of two- and three-body interactions. Applied to protein sequence data, our method reconstructs contact maps with high fidelity and outperforms state-of-the-art inverse Potts models. These results establish RBMs as a powerful and efficient tool for modeling higher-order structure in high-dimensional categorical data.