Who's the Leader? Analyzing Novice Workflows in LLM-Assisted Debugging of Machine Learning Code

TL;DR

This study examines how novice machine learning engineers interact with ChatGPT during debugging, revealing reliance patterns and cognitive challenges that impact learning and task performance.

Contribution

The paper provides a formative analysis of novice workflows with LLMs in ML debugging, highlighting reliance behaviors and proposing interaction improvements.

Findings

Novices tend to lead or be led by the LLM, affecting reliance levels.

Interactions influence over- and under-reliance on LLM outputs.

Implications for novice learning and LLM interaction design.

Abstract

While LLMs are often touted as tools for democratizing specialized knowledge to beginners, their actual effectiveness for improving task performance and learning is still an open question. It is known that novices engage with LLMs differently from experts, with prior studies reporting meta-cognitive pitfalls that affect novices' ability to verify outputs and prompt effectively. We focus on a task domain, machine learning (ML), which embodies both high complexity and low verifiability to understand the impact of LLM assistance on novices. Provided a buggy ML script and open access to ChatGPT, we conduct a formative study with eight novice ML engineers to understand their reliance on, interactions with, and perceptions of the LLM. We find that user actions can be roughly categorized into leading the LLM and led-by the LLM, and further investigate how they affect reliance outcomes like over- and under-reliance. These results have implications on novices' cognitive engagement in LLM-assisted tasks and potential negative effects on downstream learning. Lastly, we pose potential augmentations to the novice-LLM interaction paradigm to promote cognitive engagement.