What You Need is What You Get: Theory of Mind for an LLM-Based Code Understanding Assistant

TL;DR

This paper presents a personalized LLM-based conversational assistant that infers user mental states to improve code understanding support for novices, addressing barriers in natural language description, interpretation, and prompt refinement.

Contribution

It introduces a novel approach that personalizes code understanding assistance by modeling user mental states, enhancing interaction quality for novice developers.

Findings

Participants found the assistant helpful for understanding code.

The approach improved user satisfaction and perceived usefulness.

Insights guide future development of personalized LLM-based coding tools.

Abstract

A growing number of tools have used Large Language Models (LLMs) to support developers' code understanding. However, developers still face several barriers to using such tools, including challenges in describing their intent in natural language, interpreting the tool outcome, and refining an effective prompt to obtain useful information. In this study, we designed an LLM-based conversational assistant that provides a personalized interaction based on inferred user mental state (e.g., background knowledge and experience). We evaluate the approach in a within-subject study with fourteen novices to capture their perceptions and preferences. Our results provide insights for researchers and tool builders who want to create or improve LLM-based conversational assistants to support novices in code understanding.