LLM-Assisted Repository-Level Generation with Structured Spec-Driven Engineering

TL;DR

This paper introduces structured spec-driven engineering (SSDE), a new paradigm that uses structured specifications to improve large language model-based repository-level code generation, enhancing quality and verifiability.

Contribution

The paper proposes SSDE, demonstrating its potential through a pilot study on MVC logic generation with multiple LLMs, and discusses future directions.

Findings

Structured specifications improve LLM repository-level code generation.

Pilot study shows feasibility across three software systems and five LLMs.

SSDE offers enhanced verifiability and quality in code generation.

Abstract

State-of-the-art Large Language Models (LLMs) excel in code generation at the function level. However, the output quality significantly declines when scaling to repository-level systems. Current workflows relying only on natural language prompts suffer from inherent ambiguity and a lack of verifiability. To address this, we propose structured spec-driven engineering (SSDE), a paradigm that leverages structured artifacts to guide LLM generation. We argue that structured specifications as LLM inputs make high-quality, repository-level code generation a tangible goal, while at the same time offering superior verifiability, leading to significant potential for improvement. We first investigate the feasibility of this vision through a pilot study generating Model-View-Controller (MVC) business logic for three software systems using five LLMs, and then highlight the potential, challenges, and future roadmap for SSDE.