A Multi-Criteria Automated MLOps Pipeline for Cost-Effective Cloud-Based Classifier Retraining in Response to Data Distribution Shifts

TL;DR

This paper introduces an automated MLOps pipeline that detects data distribution shifts and retrains neural network classifiers efficiently, improving model robustness and accuracy in dynamic environments.

Contribution

The work presents a novel multi-criteria statistical approach for automated detection of distribution shifts and selective retraining in MLOps pipelines.

Findings

Significant accuracy improvements over traditional retraining methods

Efficient resource utilization through selective retraining

Robustness enhancement in dynamic data environments

Abstract

The performance of machine learning (ML) models often deteriorates when the underlying data distribution changes over time, a phenomenon known as data distribution drift. When this happens, ML models need to be retrained and redeployed. ML Operations (MLOps) is often manual, i.e., humans trigger the process of model retraining and redeployment. In this work, we present an automated MLOps pipeline designed to address neural network classifier retraining in response to significant data distribution changes. Our MLOps pipeline employs multi-criteria statistical techniques to detect distribution shifts and triggers model updates only when necessary, ensuring computational efficiency and resource optimization. We demonstrate the effectiveness of our framework through experiments on several benchmark anomaly detection data sets, showing significant improvements in model accuracy and robustness compared to traditional retraining strategies. Our work provides a foundation for deploying more reliable and adaptive ML systems in dynamic real-world settings, where data distribution changes are common.