A quantitative framework for evaluating architectural patterns in ML systems

TL;DR

This paper presents a quantitative framework to evaluate architectural patterns in machine learning systems, focusing on scalability and performance to aid architects in selecting optimal designs.

Contribution

It introduces a systematic evaluation process for architectural patterns in ML systems, emphasizing measurable metrics for better decision-making.

Findings

Framework effectively assesses architectural patterns based on scalability and performance

Application through a case study demonstrates practical utility

Enables objective comparison and selection of ML system architectures

Abstract

Contemporary intelligent systems incorporate software components, including machine learning components. As they grow in complexity and data volume such machine learning systems face unique quality challenges like scalability and performance. To overcome them, engineers may often use specific architectural patterns, however their impact on ML systems is difficult to quantify. The effect of software architecture on traditional systems is well studied, however more work is needed in the area of machine learning systems. This study proposes a framework for quantitative assessment of architectural patterns in ML systems, focusing on scalability and performance metrics for cost-effective CPU-based inference. We integrate these metrics into a systematic evaluation process for selection of architectural patterns and demonstrate its application through a case study. The approach shown in the paper should enable software architects to objectively analyze and select optimal patterns, addressing key challenges in ML system design.