Reusable MLOps: Reusable Deployment, Reusable Infrastructure and Hot-Swappable Machine Learning models and services

TL;DR

This paper introduces Reusable MLOps, a novel approach enabling the reuse of deployment infrastructure and hot-swapping models without downtime, streamlining AI/ML operationalization and reducing development effort.

Contribution

It presents the Acumos AI platform and a new concept of Reusable MLOps, facilitating continuous model training and deployment with reusable infrastructure and hot-swappable models.

Findings

Demonstrates the Acumos platform's capabilities for reusable deployment.

Shows how hot-swapping models reduces downtime and operational costs.

Validates the approach with practical examples and case studies.

Abstract

Although Machine Learning model building has become increasingly accessible due to a plethora of tools, libraries and algorithms being available freely, easy operationalization of these models is still a problem. It requires considerable expertise in data engineering, software development, cloud and DevOps. It also requires planning, agreement, and vision of how the model is going to be used by the business applications once it is in production, how it is going to be continuously trained on fresh incoming data, and how and when a newer model would replace an existing model. This leads to developers and data scientists working in silos and making suboptimal decisions. It also leads to wasted time and effort. We introduce the Acumos AI platform we developed and we demonstrate some unique novel capabilities that the Acumos model runner possesses, that can help solve the above problems. We introduce a new sustainable concept in the field of AI/ML operations - called Reusable MLOps - where we reuse the existing deployment and infrastructure to serve new models by hot-swapping them without tearing down the infrastructure or the microservice, thus achieving reusable deployment and operations for AI/ML models while still having continuously trained models in production.