Mechanistic Neural Networks for Scientific Machine Learning

TL;DR

Mechanistic Neural Networks integrate a novel Mechanistic Block and a Relaxed Linear Programming Solver to improve interpretability, efficiency, and scalability in scientific machine learning tasks such as equation discovery and dynamic systems modeling.

Contribution

The paper introduces Mechanistic Neural Networks with a new Mechanistic Block and NeuRLP, advancing the ability to learn governing equations and dynamics explicitly and efficiently.

Findings

Outperforms state-of-the-art methods in scientific data analysis.

Enables scalable GPU-based processing for differential equations.

Provides interpretable models of complex scientific phenomena.

Abstract

This paper presents Mechanistic Neural Networks, a neural network design for machine learning applications in the sciences. It incorporates a new Mechanistic Block in standard architectures to explicitly learn governing differential equations as representations, revealing the underlying dynamics of data and enhancing interpretability and efficiency in data modeling. Central to our approach is a novel Relaxed Linear Programming Solver (NeuRLP) inspired by a technique that reduces solving linear ODEs to solving linear programs. This integrates well with neural networks and surpasses the limitations of traditional ODE solvers enabling scalable GPU parallel processing. Overall, Mechanistic Neural Networks demonstrate their versatility for scientific machine learning applications, adeptly managing tasks from equation discovery to dynamic systems modeling. We prove their comprehensive capabilities in analyzing and interpreting complex scientific data across various applications, showing significant performance against specialized state-of-the-art methods.