LLM-based Automated Architecture View Generation: Where Are We Now?

TL;DR

This paper empirically evaluates the effectiveness of large language models and agentic approaches in automatically generating software architecture views from source code, highlighting their capabilities and current limitations.

Contribution

It provides a comprehensive empirical analysis of LLMs and agentic methods for architecture view generation, comparing prompting techniques and evaluating performance on real-world repositories.

Findings

Few-shot prompting reduces clarity failures by 9.2%.

Agentic approaches outperform general-purpose agents in clarity and detail.

Generated views often mismatch granularity, operating at code rather than architectural level.

Abstract

Architecture views are essential for software architecture documentation, yet their manual creation is labor intensive and often leads to outdated artifacts. As systems grow in complexity, the automated generation of views from source code becomes increasingly valuable. Goal: We empirically evaluate the ability of LLMs and agentic approaches to generate architecture views from source code. Method: We analyze 340 open-source repositories across 13 experimental configurations using 3 LLMs with 3 prompting techniques and 2 agentic approaches, yielding 4,137 generated views. We evaluate the generated views by comparing them with the ground-truth using a combination of automated metrics complemented by human evaluations. Results: Prompting strategies offer marginal improvements. Few-shot prompting reduces clarity failures by 9.2% compared to zero-shot baselines. The custom agentic approach consistently outperforms the general-purpose agent, achieving the best clarity (22.6% failure rate) and level-of-detail success (50%). Conclusions: LLM and agentic approaches demonstrate capabilities in generating syntactically valid architecture views. However, they consistently exhibit granularity mismatches, operating at the code level rather than architectural abstractions. This suggests that there is still a need for human expertise, positioning LLMs and agents as assistive tools rather than autonomous architects.