MEP-Net: Generating Solutions to Scientific Problems with Limited Knowledge by Maximum Entropy Principle

TL;DR

This paper introduces MEP-Net, a neural network architecture that integrates the maximum entropy principle to generate probability distributions from limited information, with applications in biochemical systems.

Contribution

The paper presents a novel neural network model combining MEP with deep learning to infer distributions from moment constraints, justified by large deviations theory.

Findings

Effective in modeling biochemical reaction networks

Generates complex distributions from limited data

Theoretically justified for non-equilibrium systems

Abstract

Maximum entropy principle (MEP) offers an effective and unbiased approach to inferring unknown probability distributions when faced with incomplete information, while neural networks provide the flexibility to learn complex distributions from data. This paper proposes a novel neural network architecture, the MEP-Net, which combines the MEP with neural networks to generate probability distributions from moment constraints. We also provide a comprehensive overview of the fundamentals of the maximum entropy principle, its mathematical formulations, and a rigorous justification for its applicability for non-equilibrium systems based on the large deviations principle. Through fruitful numerical experiments, we demonstrate that the MEP-Net can be particularly useful in modeling the evolution of probability distributions in biochemical reaction networks and in generating complex distributions from data.