Defining Foundation Models for Computational Science: A Call for Clarity and Rigor

TL;DR

This paper proposes a formal definition of foundation models in computational science, emphasizing clarity, and introduces the Data-Driven Finite Element Method (DD-FEM) to bridge traditional numerical methods with AI for scalable, adaptable, and physics-consistent models.

Contribution

It establishes a rigorous definition of foundation models in computational science and introduces DD-FEM, integrating classical finite element methods with data-driven learning.

Findings

DD-FEM enhances scalability and adaptability in computational models.

The formal definition clarifies the core attributes of foundation models.

Bridges traditional numerical methods with modern AI paradigms.

Abstract

The widespread success of foundation models in natural language processing and computer vision has inspired researchers to extend the concept to scientific machine learning and computational science. However, this position paper argues that as the term "foundation model" is an evolving concept, its application in computational science is increasingly used without a universally accepted definition, potentially creating confusion and diluting its precise scientific meaning. In this paper, we address this gap by proposing a formal definition of foundation models in computational science, grounded in the core values of generality, reusability, and scalability. We articulate a set of essential and desirable characteristics that such models must exhibit, drawing parallels with traditional foundational methods, like the finite element and finite volume methods. Furthermore, we introduce the Data-Driven Finite Element Method (DD-FEM), a framework that fuses the modular structure of classical FEM with the representational power of data-driven learning. We demonstrate how DD-FEM addresses many of the key challenges in realizing foundation models for computational science, including scalability, adaptability, and physics consistency. By bridging traditional numerical methods with modern AI paradigms, this work provides a rigorous foundation for evaluating and developing novel approaches toward future foundation models in computational science.