Towards MLOps: A DevOps Tools Recommender System for Machine Learning System

TL;DR

This paper proposes a framework for recommending open-source tools for MLOps pipelines based on project context, using machine learning models to improve tool selection accuracy.

Contribution

It introduces a novel recommendation system framework that processes contextual information to suggest suitable MLOps tools, evaluated with multiple machine learning approaches.

Findings

Random forest achieved highest F-score of 0.66

The framework effectively recommends relevant toolchains based on project context

Machine learning models can improve tool selection in MLOps workflows

Abstract

Applying DevOps practices to machine learning system is termed as MLOps and machine learning systems evolve on new data unlike traditional systems on requirements. The objective of MLOps is to establish a connection between different open-source tools to construct a pipeline that can automatically perform steps to construct a dataset, train the machine learning model and deploy the model to the production as well as store different versions of model and dataset. Benefits of MLOps is to make sure the fast delivery of the new trained models to the production to have accurate results. Furthermore, MLOps practice impacts the overall quality of the software products and is completely dependent on open-source tools and selection of relevant open-source tools is considered as challenged while a generalized method to select an appropriate open-source tools is desirable. In this paper, we present a framework for recommendation system that processes the contextual information (e.g., nature of data, type of the data) of the machine learning project and recommends a relevant toolchain (tech-stack) for the operationalization of machine learning systems. To check the applicability of the proposed framework, four different approaches i.e., rule-based, random forest, decision trees and k-nearest neighbors were investigated where precision, recall and f-score is measured, the random forest out classed other approaches with highest f-score value of 0.66.