Optimal Linear Baseline Models for Scientific Machine Learning

TL;DR

This paper develops a theoretical framework for linear models in scientific machine learning, providing optimal mappings and benchmarks for interpretability and understanding complex physical data relationships.

Contribution

It introduces a unified approach for analyzing linear encoder-decoder models, deriving closed-form solutions that handle rank deficiencies and generalize existing methods.

Findings

Derived closed-form optimal linear mappings for modeling and inverse problems.

Validated theoretical results with experiments in biomedical imaging, finance, and fluid dynamics.

Provided a robust baseline for benchmarking neural network models in scientific applications.

Abstract

Across scientific domains, a fundamental challenge is to characterize and compute the mappings from underlying physical processes to observed signals and measurements. While nonlinear neural networks have achieved considerable success, they remain theoretically opaque, which hinders adoption in contexts where interpretability is paramount. In contrast, linear neural networks serve as a simple yet effective foundation for gaining insight into these complex relationships. In this work, we develop a unified theoretical framework for analyzing linear encoder-decoder architectures through the lens of Bayes risk minimization for solving data-driven scientific machine learning problems. We derive closed-form, rank-constrained linear and affine linear optimal mappings for forward modeling and inverse recovery tasks. Our results generalize existing formulations by accommodating rank-deficiencies in data, forward operators, and measurement processes. We validate our theoretical results by conducting numerical experiments on datasets from simple biomedical imaging, financial factor analysis, and simulations involving nonlinear fluid dynamics via the shallow water equations. This work provides a robust baseline for understanding and benchmarking learned neural network models for scientific machine learning problems.