Practice Less, Explain More: LLM-Supported Self-Explanation Improves Explanation Quality on Transfer Problems in Calculus

TL;DR

This study shows that LLM-supported open-ended self-explanation enhances explanation quality on transfer problems in calculus, even with fewer practice problems, compared to other methods.

Contribution

It demonstrates that LLM-supported open-ended self-explanation improves transfer explanation quality in calculus learning environments.

Findings

Open-ended self-explanation led to higher-quality explanations on transfer problems.

Learners in the open-ended condition completed fewer practice problems.

Effects on explanation quality were significant even with fewer practice problems.

Abstract

We conducted a between-subjects experiment (N=92) comparing three conditions in a calculus learning environment: no self-explanation (control), menu-based self-explanation, and open-ended self-explanation with LLM-generated feedback. All conditions showed positive learning gains within a fixed 60-minute practice session, with no significant between-condition differences in post-test performance. On transfer questions, the open-ended condition produced significantly higher-quality explanations than control on "Not Enough Information" (NEI) problems ($\beta$=+11.9 percentage points, $p$=.030), though the corresponding NEI multiple-choice accuracy advantage was not significant ($p$=.183). Moreover, across all post-test open-ended explanations, the open-ended condition showed a marginally significant advantage ($\beta$=+7.3%, $p$=.057). These findings suggest that LLM-supported open-ended self-explanation can improve explanation quality on NEI transfer problems, with weaker evidence across broader transfer explanation measures. Notably, these effects emerged even though learners in the open-ended condition completed substantially fewer practice problems within the same practice time.