Multimodal Scientific Learning Beyond Diffusions and Flows

TL;DR

This paper advocates for using Mixture Density Networks (MDNs) as an efficient, interpretable alternative to diffusion and flow-based models for multimodal uncertainty quantification in scientific machine learning, especially with limited data.

Contribution

It introduces a unified probabilistic framework highlighting the advantages of MDNs over implicit models in capturing multimodal scientific phenomena.

Findings

MDNs outperform implicit models in data efficiency and generalization.

MDNs provide better interpretability for scientific problems.

Empirical results show MDNs effectively recover multiple solution modes.

Abstract

Scientific machine learning (SciML) increasingly requires models that capture multimodal conditional uncertainty arising from ill-posed inverse problems, multistability, and chaotic dynamics. While recent work has favored highly expressive implicit generative models such as diffusion and flow-based methods, these approaches are often data-hungry, computationally costly, and misaligned with the structured solution spaces frequently found in scientific problems. We demonstrate that Mixture Density Networks (MDNs) provide a principled yet largely overlooked alternative for multimodal uncertainty quantification in SciML. As explicit parametric density estimators, MDNs impose an inductive bias tailored to low-dimensional, multimodal physics, enabling direct global allocation of probability mass across distinct solution branches. This structure delivers strong data efficiency, allowing reliable recovery of separated modes in regimes where scientific data is scarce. We formalize these insights through a unified probabilistic framework contrasting explicit and implicit distribution networks, and demonstrate empirically that MDNs achieve superior generalization, interpretability, and sample efficiency across a range of inverse, multistable, and chaotic scientific regression tasks.