User Misconceptions of LLM-Based Conversational Programming Assistants

TL;DR

This paper investigates user misconceptions about LLM-based programming assistants, highlighting the need for clearer communication of tool capabilities to prevent over-reliance and improve programming practices.

Contribution

It provides a detailed characterization of misconceptions through qualitative analysis of real user conversations, emphasizing the importance of transparent tool communication.

Findings

Users have misplaced expectations about web access and code execution.

Misconceptions can lead to over-reliance and unproductive practices.

Deeper issues involve understanding debugging and validation information.

Abstract

Programming assistants powered by large language models (LLMs) have become widely available, with conversational assistants like ChatGPT particularly accessible to novice programmers. However, varied tool capabilities and inconsistent availability of extensions (web search, code execution, retrieval-augmented generation) create opportunities for user misconceptions that may lead to over-reliance, unproductive practices, or insufficient quality control. We characterize misconceptions that users of conversational LLM-based assistants may have in programming contexts through a two-phase approach: first brainstorming and cataloging potential misconceptions, then conducting qualitative analysis of Python-programming conversations from the WildChat dataset. We find evidence that users have misplaced expectations about features like web access, code execution, and non-text outputs. We also note the potential for deeper conceptual issues around information requirements for debugging, validation, and optimization. Our findings reinforce the need for LLM-based tools to more clearly communicate their capabilities to users and empirically ground aspects that require clarification in programming contexts.