A Methodology for Deriving Evaluation Criteria for Software Solutions

TL;DR

This paper presents a formal methodology for deriving tailored evaluation criteria for software solutions, considering technical fit and strategic alignment, exemplified on Machine-Learning-as-a-Service platforms for SMEs.

Contribution

It introduces a three-layer formal model to generate individualized evaluation criteria from a general software-agnostic catalogue.

Findings

The methodology effectively encodes domain knowledge and industry context.

It produces refined, tailored criteria for specific software solutions.

Demonstrated on MaaS platforms for SMEs.

Abstract

Finding a suited software solution for a company poses a resource-intensive task in an ever-widening market. Software should solve the technical task at hand as perfectly as possible and, at the same time, match the company strategy. Based on these two dimensions, domain knowledge and industry context, we propose a methodology for deriving individually tailored evaluation criteria for software solutions to make them assessable. The approach is formalized as a three-layer model, that ensures the encoding of said dimensions, where each layer holds a more refined and individualized criteria list, starting from a general softwareagnostic catalogue we composed. Finally, we exemplarily demonstrate our method for Machine-Learning-as-a-Service platforms (MaaS) for small and medium-sized enterprises (SME).