An Empirical Study on Low-Code Programming using Traditional vs Large Language Model Support

TL;DR

This study empirically compares traditional low-code programming tools with those supported by large language models, analyzing developer discussions to understand their differences, similarities, and impact on software development.

Contribution

It provides a comprehensive analysis of traditional and LLM-based low-code programming, highlighting their distinct features, limitations, and integration in the software development lifecycle.

Findings

Traditional and LLM-based LCP share common usage scenarios.

Significant differences exist in scope and limitations, especially during implementation.

LLMs influence and enhance LCP through integration with VPLs and LLM Agents.

Abstract

Low-code programming (LCP) refers to programming using models at higher levels of abstraction, resulting in less manual and more efficient programming, and reduced learning effort for amateur developers. Many LCP tools have rapidly evolved and have benefited from the concepts of visual programming languages (VPLs) and programming by demonstration (PBD). With the huge increase in interest in using large language models (LLMs) in software engineering, LLM-based LCP has begun to become increasingly important. However, the technical principles and application scenarios of traditional approaches to LCP and LLM-based LCP are significantly different. Understanding these key differences and characteristics in the application of the two approaches to LCP by users is crucial for LCP providers in improving existing and developing new LCP tools and in better assisting users in choosing the appropriate LCP technology. We conducted an empirical study of both traditional LCP and LLM-based LCP. We analyzed developers' discussions on Stack Overflow (SO) over the past three years and then explored the similarities and differences between traditional LCP and LLM-based LCP features and developer feedback. Our findings reveal that while traditional LCP and LLM-based LCP share common primary usage scenarios, they significantly differ in scope, limitations, and usage throughout the software development lifecycle, particularly during the implementation phase. We also examine how LLMs impact and integrate with LCP, discussing the latest technological developments in LLM-based LCP, such as its integration with VPLs and the application of LLM Agents in software engineering.