GenAIOps for GenAI Model-Agility
Ken Ueno, Makoto Kogo, Hiromi Kawatsu, Yohsuke Uchiumi, Michiaki, Tatsubori

TL;DR
This paper explores GenAIOps, a methodology for enhancing the agility of generative AI applications by managing model changes, and evaluates prompt tuning techniques for maintaining application quality amidst foundation model updates.
Contribution
It introduces the concept of GenAI Model-agility and proposes a methodology for managing model changes, including an analysis of prompt tuning effectiveness and limitations.
Findings
Prompt tuning can mitigate quality degradation due to model updates.
Prompt tuning effectiveness varies across different tools and scenarios.
Identifies limitations of current prompt tuning approaches.
Abstract
AI-agility, with which an organization can be quickly adapted to its business priorities, is desired even for the development and operations of generative AI (GenAI) applications. Especially in this paper, we discuss so-called GenAI Model-agility, which we define as the readiness to be flexibly adapted to base foundation models as diverse as the model providers and versions. First, for handling issues specific to generative AI, we first define a methodology of GenAI application development and operations, as GenAIOps, to identify the problem of application quality degradation caused by changes to the underlying foundation models. We study prompt tuning technologies, which look promising to address this problem, and discuss their effectiveness and limitations through case studies using existing tools.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSemantic Web and Ontologies
MethodsBalanced Selection
