From Cool Demos to Production-Ready FMware: Core Challenges and a Technology Roadmap
Gopi Krishnan Rajbahadur, Gustavo A. Oliva, Dayi Lin, Jiho Shin, Ahmed E. Hassan

TL;DR
This paper identifies key challenges and proposes a roadmap for transitioning foundation model-based systems from demos to scalable, production-ready solutions, emphasizing reliability, scalability, and compliance.
Contribution
It provides a comprehensive analysis of technical and operational challenges in productionizing FMware and offers strategic guidance based on industry experience and literature.
Findings
Identified critical issues in FMware deployment including data alignment and system testing.
Highlighted the importance of memory management and observability in production systems.
Provided a technology roadmap for scalable FMware development.
Abstract
The rapid expansion of foundation models (FMs), such as large language models (LLMs), has given rise to FMware, software systems that integrate FM(s) as core components. While building demonstration-level FMware is relatively straightforward, transitioning to production-ready systems presents numerous challenges, including reliability, high implementation costs, scalability, and compliance with privacy regulations. Our paper conducts a semi-structured thematic synthesis to identify key challenges in productionizing FMware across diverse data sources, including our industry experience developing FMArts, a FMware lifecycle engineering platform, and its integration into Huawei Cloud; grey literature; academic publications; hands-on involvement in the Open Platform for Enterprise AI (OPEA); organizing the AIware conference and bootcamp; and co-leading the ISO SPDX SBOM working group on AI…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
