Software Architecture Meets LLMs: A Systematic Literature Review

TL;DR

This systematic review examines how Large Language Models are applied in software architecture, highlighting their current uses, effectiveness, challenges, and areas needing further research.

Contribution

It provides a comprehensive overview of 18 studies on LLMs in software architecture, identifying trends, gaps, and future research directions.

Findings

LLMs are increasingly used for various architecture tasks.

They often outperform traditional baselines.

Certain areas like code generation and conformance checking are underexplored.

Abstract

Large Language Models (LLMs) are used for many different software engineering tasks. In software architecture, they have been applied to tasks such as classification of design decisions, detection of design patterns, and generation of software architecture design from requirements. However, there is little overview on how well they work, what challenges exist, and what open problems remain. In this paper, we present a systematic literature review on the use of LLMs in software architecture. We analyze 18 research articles to answer five research questions, such as which software architecture tasks LLMs are used for, how much automation they provide, which models and techniques are used, and how these approaches are evaluated. Our findings show that while LLMs are increasingly applied to a variety of software architecture tasks and often outperform baselines, some areas, such as generating source code from architectural design, cloud-native computing and architecture, and checking conformance remain underexplored. Although current approaches mostly use simple prompting techniques, we identify a growing research interest in refining LLM-based approaches by integrating advanced techniques.