MLmisFinder: A Specification and Detection Approach of Machine Learning Service Misuses

TL;DR

MLmisFinder is an automatic detection approach that identifies improper uses of machine learning services in software systems, improving quality and maintainability by accurately detecting misuses across numerous systems.

Contribution

It introduces a novel metamodel and rule-based algorithms for detecting seven types of ML service misuses, outperforming existing baselines.

Findings

Achieves 96.7% precision and 97% recall in detection.

Effectively scales to 817 systems, revealing widespread misuses.

Highlights common misuse areas like data drift monitoring.

Abstract

Machine Learning (ML) cloud services, offered by leading providers such as Amazon, Google, and Microsoft, enable the integration of ML components into software systems without building models from scratch. However, the rapid adoption of ML services, coupled with the growing complexity of business requirements, has led to widespread misuses, compromising the quality, maintainability, and evolution of ML service-based systems. Though prior research has studied patterns and antipatterns in service-based and ML-based systems separately, automatic detection of ML service misuses remains a challenge. In this paper, we propose MLmisFinder, an automatic approach to detect ML service misuses in software systems, aiming to identify instances of improper use of ML services to help developers properly integrate ML components in ML service-based systems. We propose a metamodel that captures the data needed to detect misuses in ML service-based systems and apply a set of rule-based detection algorithms for seven misuse types. We evaluated MLmisFinder on 107 software systems collected from open-source GitHub repositories and compared it with a state-of-the-art baseline. Our results show that MLmisFinder effectively detects ML service misuses, achieving an average precision of 96.7\% and recall of 97\%, outperforming the state-of-the-art baseline. MLmisFinder also scaled efficiently to detect misuses across 817 ML service-based systems and revealed that such misuses are widespread, especially in areas such as data drift monitoring and schema validation.