SWITCH: An Exemplar for Evaluating Self-Adaptive ML-Enabled Systems

TL;DR

SWITCH is a comprehensive platform that enables dynamic ML model switching at runtime to improve QoS in self-adaptive ML-enabled systems, demonstrated through an object detection case study.

Contribution

This paper introduces SWITCH, a flexible web service platform for evaluating and applying self-adaptive ML model switching strategies in real-time systems.

Findings

Improved QoS demonstrated in object detection case study

Enhanced system observability with real-time dashboard

Flexible platform for research on adaptive ML strategies

Abstract

Addressing runtime uncertainties in Machine Learning-Enabled Systems (MLS) is crucial for maintaining Quality of Service (QoS). The Machine Learning Model Balancer is a concept that addresses these uncertainties by facilitating dynamic ML model switching, showing promise in improving QoS in MLS. Leveraging this concept, this paper introduces SWITCH, an exemplar developed to enhance self-adaptive capabilities in such systems through dynamic model switching in runtime. SWITCH is designed as a comprehensive web service catering to a broad range of ML scenarios, with its implementation demonstrated through an object detection use case. SWITCH provides researchers with a flexible platform to apply and evaluate their ML model switching strategies, aiming to enhance QoS in MLS. SWITCH features advanced input handling, real-time data processing, and logging for adaptation metrics supplemented with an interactive real-time dashboard for enhancing system observability. This paper details SWITCH's architecture, self-adaptation strategies through ML model switching, and its empirical validation through a case study, illustrating its potential to improve QoS in MLS. By enabling a hands-on approach to explore adaptive behaviors in ML systems, SWITCH contributes a valuable tool to the SEAMS community for research into self-adaptive mechanisms for MLS and their practical applications.