ML-On-Rails: Safeguarding Machine Learning Models in Software Systems A Case Study

TL;DR

This paper presents ML-On-Rails, a protocol to enhance the safety, security, and transparency of machine learning models in production systems, demonstrated through a real-world case study.

Contribution

Introduction of ML-On-Rails, a protocol for safeguarding ML models and establishing clear interfaces between ML providers and consumers in software systems.

Findings

ML-On-Rails improves model robustness in production environments.

The protocol facilitates better communication between ML providers and users.

Case study confirms the protocol's effectiveness in real-world application.

Abstract

Machine learning (ML), especially with the emergence of large language models (LLMs), has significantly transformed various industries. However, the transition from ML model prototyping to production use within software systems presents several challenges. These challenges primarily revolve around ensuring safety, security, and transparency, subsequently influencing the overall robustness and trustworthiness of ML models. In this paper, we introduce ML-On-Rails, a protocol designed to safeguard ML models, establish a well-defined endpoint interface for different ML tasks, and clear communication between ML providers and ML consumers (software engineers). ML-On-Rails enhances the robustness of ML models via incorporating detection capabilities to identify unique challenges specific to production ML. We evaluated the ML-On-Rails protocol through a real-world case study of the MoveReminder application. Through this evaluation, we emphasize the importance of safeguarding ML models in production.