Revisiting the Impact of Pursuing Modularity for Code Generation

TL;DR

This paper investigates whether modular programming enhances code generation performance in large language models, introducing a new metric and finding that modularity is not a key factor for improving model outputs.

Contribution

It introduces a novel metric to quantify modularity and challenges the conventional belief that modular code improves LLM-based code generation.

Findings

Modularity is not a significant factor in code generation performance.

A new metric for measuring modularity in code is proposed.

LLMs do not prefer modular code over non-modular code.

Abstract

Modular programming, which aims to construct the final program by integrating smaller, independent building blocks, has been regarded as a desirable practice in software development. However, with the rise of recent code generation agents built upon large language models (LLMs), a question emerges: is this traditional practice equally effective for these new tools? In this work, we assess the impact of modularity in code generation by introducing a novel metric for its quantitative measurement. Surprisingly, unlike conventional wisdom on the topic, we find that modularity is not a core factor for improving the performance of code generation models. We also explore potential explanations for why LLMs do not exhibit a preference for modular code compared to non-modular code.