Differentiable neural network representation of multi-well, locally-convex potentials

TL;DR

This paper introduces a differentiable, convex neural network model called LSE-ICNN for representing multi-well potentials, enabling smooth, gradient-based analysis of complex multimodal phenomena across various scientific domains.

Contribution

The paper proposes the LSE-ICNN, a novel neural network formulation that automatically discovers modes and transition scales, providing a flexible, convex, and differentiable surrogate for multi-well potentials.

Findings

Successfully models phase transformations and elastic instabilities.

Captures complex multimodal landscapes with smooth gradients.

Applicable across physics, chemistry, and biology domains.

Abstract

Multi-well potentials are ubiquitous in science, modeling phenomena such as phase transitions, dynamic instabilities, and multimodal behavior across physics, chemistry, and biology. In contrast to non-smooth minimum-of-mixture representations, we propose a differentiable and convex formulation based on a log-sum-exponential (LSE) mixture of input convex neural network (ICNN) modes. This log-sum-exponential input convex neural network (LSE-ICNN) provides a smooth surrogate that retains convexity within basins and allows for gradient-based learning and inference. A key feature of the LSE-ICNN is its ability to automatically discover both the number of modes and the scale of transitions through sparse regression, enabling adaptive and parsimonious modeling. We demonstrate the versatility of the LSE-ICNN across diverse domains, including mechanochemical phase transformations, microstructural elastic instabilities, conservative biological gene circuits, and variational inference for multimodal probability distributions. These examples highlight the effectiveness of the LSE-ICNN in capturing complex multimodal landscapes while preserving differentiability, making it broadly applicable in data-driven modeling, optimization, and physical simulation.