Training Software Engineers for Qualitative Evaluation of Software Architecture

TL;DR

This paper introduces a machine learning-based framework that helps novice software architects evaluate and improve system designs by analyzing architectural diagrams to identify patterns, quality attributes, and tactics.

Contribution

It presents a novel framework that uses image processing and semantic similarity to assist novices in architectural evaluation, including pattern recognition and quality attribute assessment.

Findings

Achieved 98.71% accuracy in pattern recognition

Experimental group performed 150% better in evaluations

Developed a dataset of 2,035 architectural images

Abstract

A software architect uses quality requirements to design the architecture of a system. However, it is essential to ensure that the system's final architectural design achieves the standard quality requirements. The existing architectural evaluation frameworks require basic skills and experience for practical usage, which novice software architects lack. We propose a framework that enables novice software architects to infer the system's quality requirements and tactics using the software architectural block-line diagram. The framework takes an image as input, extracts various components and connections, and maps them to viable architectural patterns, followed by identifying the system's corresponding quality attributes (QAs) and tactics. The framework includes a specifically trained machine learning model based on image processing and semantic similarity methods to assist software architects in evaluating a given design by a) evaluating an input architectural design based on the architectural patterns present in it, b) lists out the strengths and weaknesses of the design in terms of QAs, c) recommends the necessary architectural tactics that can be embedded in the design to achieve the lacking QAs. To train our framework, we developed a dataset of 2,035 architectural images from fourteen architectural patterns such as Client-Server, Microservices, and Model View Controller, available at https://www.doi.org/10.6084/m9.figshare.14156408. The framework achieves a Correct Recognition Rate of 98.71% in identifying the architectural patterns. We evaluated the proposed framework's effectiveness and usefulness by using controlled and experimental groups, in which the experimental group performed approximately 150% better than the controlled group. The experiments were performed as a part of the Masters of Computer Science course in an Engineering Institution.