Applying empirical software engineering to software architecture: challenges and lessons learned

TL;DR

This paper discusses the integration of empirical research methods into software architecture to improve evidence-based practices, highlighting challenges and lessons learned from various empirical studies over the past 15 years.

Contribution

It promotes applying empirical paradigms to software architecture research and shares insights from experiences with different empirical methods.

Findings

Empirical methods can validate software architecture techniques.

Controlled experiments and surveys reveal practical challenges.

Lessons learned support development of well-founded architecture theories.

Abstract

In the last 15 years, software architecture has emerged as an important software engineering field for managing the development and maintenance of large, software- intensive systems. Software architecture community has developed numerous methods, techniques, and tools to support the architecture process (analysis, design, and review). Historically, most advances in software architecture have been driven by talented people and industrial experience, but there is now a growing need to systematically gather empirical evidence about the advantages or otherwise of tools and methods rather than just rely on promotional anecdotes or rhetoric. The aim of this paper is to promote and facilitate the application of the empirical paradigm to software architecture. To this end, we describe the challenges and lessons learned when assessing software architecture research that used controlled experiments, replications, expert opinion, systematic literature reviews, obser- vational studies, and surveys. Our research will support the emergence of a body of knowledge consisting of the more widely-accepted and well-formed software architecture theories.