Ensuring Robustness in ML-enabled Software Systems: A User Survey

TL;DR

This paper introduces the ML-On-Rails protocol, a comprehensive framework to improve robustness and trustworthiness of ML-enabled systems, validated through a practitioner survey highlighting current challenges and gaps.

Contribution

The paper proposes the ML-On-Rails protocol, integrating safeguards and communication mechanisms to enhance robustness in production ML systems, supported by real-world practitioner insights.

Findings

Major robustness issues identified in practice

Gaps in current solutions for ML system robustness

Survey supports the need for standardized protocols like ML-On-Rails

Abstract

Ensuring robustness in ML-enabled software systems requires addressing critical challenges, such as silent failures, out-of-distribution (OOD) data, and adversarial attacks. Traditional software engineering practices, which rely on predefined logic, are insufficient for ML components that depend on data and probabilistic decision-making. To address these challenges, we propose the ML-On-Rails protocol, a unified framework designed to enhance the robustness and trustworthiness of ML-enabled systems in production. This protocol integrates key safeguards such as OOD detection, adversarial attack detection, input validation, and explainability. It also includes a model-to-software communication framework using HTTP status codes to enhance transparency in reporting model outcomes and errors. To align our approach with real-world challenges, we conducted a practitioner survey, which revealed major robustness issues, gaps in current solutions, and highlighted how a standardised protocol such as ML-On-Rails can improve system robustness. Our findings highlight the need for more support and resources for engineers working with ML systems. Finally, we outline future directions for refining the proposed protocol, leveraging insights from the survey and real-world applications to continually enhance its effectiveness.