Exploring Influence Factors on LLM Suitability for No-Code Development of End User IoT Applications

TL;DR

This paper systematically investigates how different factors like model choice, prompt language, training data, and error feedback influence the effectiveness of LLMs in supporting non-technical users to develop IoT applications via no-code platforms.

Contribution

It provides a comprehensive experimental analysis of key factors affecting LLM performance in no-code IoT app development, guiding better integration and selection of models.

Findings

Model selection significantly impacts application quality.

Prompt language influences user interaction effectiveness.

Training data background affects LLM adaptability.

Abstract

No-Code Development Platforms (NCDPs) empower non-technical end users to build applications tailored to their specific demands without writing code. While NCDPs lower technical barriers, users still require some technical knowledge, e.g., to structure process steps or define event-action rules. Large Language Models (LLMs) offer a promising solution to further reduce technical requirements by supporting natural language interaction and dynamic code generation. By integrating LLM, NCDPs can be more accessible to non-technical users, enabling application development truly without requiring any technical expertise. Despite growing interest in LLM-powered NCDPs, a systematic investigation into the factors influencing LLM suitability and performance remains absent. Understanding these factors is critical to effectively leveraging LLMs capabilities and maximizing their impact. In this paper, we investigate key factors influencing the effectiveness of LLMs in supporting end-user application development within NCDPs. By conducting comprehensive experiments, we evaluate the impact of four key factors, i.e., model selection, prompt language, training data background, and an error-informed few-shot setup, on the quality of generated applications. Specifically, we selected a range of LLMs based on their architecture, scale, design focus, and training data, and evaluated them across four real-world smart home automation scenarios implemented on a representative open-source LLM-powered NCDP. Our findings offer practical insights into how LLMs can be effectively integrated into NCDPs, informing both platform design and the selection of suitable LLMs for end-user application development.