Foundation Model Engineering: Engineering Foundation Models Just as Engineering Software

TL;DR

This paper advocates for a new engineering discipline focused on managing the complexity of Foundation Models by developing structured, automated, and unified tools and methodologies, inspired by traditional software engineering principles.

Contribution

It introduces the concept of Foundation Model engineering as a strategic approach to address the FM crisis with principled, automated, and unified management interfaces.

Findings

Proposes declarative programming interfaces for FMs

Highlights the need for structured FM development methodologies

Suggests new research directions in software engineering for FMs

Abstract

By treating data and models as the source code, Foundation Models (FMs) become a new type of software. Mirroring the concept of software crisis, the increasing complexity of FMs making FM crisis a tangible concern in the coming decade, appealing for new theories and methodologies from the field of software engineering. In this paper, we outline our vision of introducing Foundation Model (FM) engineering, a strategic response to the anticipated FM crisis with principled engineering methodologies. FM engineering aims to mitigate potential issues in FM development and application through the introduction of declarative, automated, and unified programming interfaces for both data and model management, reducing the complexities involved in working with FMs by providing a more structured and intuitive process for developers. Through the establishment of FM engineering, we aim to provide a robust, automated, and extensible framework that addresses the imminent challenges, and discovering new research opportunities for the software engineering field.