Tutorly: Turning Programming Videos Into Apprenticeship Learning Environments with LLMs

TL;DR

Tutorly transforms online programming videos into interactive, personalized tutoring experiences using LLMs and learner modeling, significantly improving learners' performance in exploratory data analysis tasks.

Contribution

It introduces Tutorly, a JupyterLab plugin that creates personalized apprenticeship learning environments from programming videos with LLMs and learner models.

Findings

Performance improved from 61.9% to 76.6% after using Tutorly.

Learners engaged more effectively with personalized guidance.

Demonstrates potential for scalable, interactive programming education.

Abstract

Online programming videos, including tutorials and streamcasts, are widely popular and contain a wealth of expert knowledge. However, effectively utilizing these resources to achieve targeted learning goals can be challenging. Unlike direct tutoring, video content lacks tailored guidance based on individual learning paces, personalized feedback, and interactive engagement necessary for support and monitoring. Our work transforms programming videos into one-on-one tutoring experiences using the cognitive apprenticeship framework. Tutorly, developed as a JupyterLab Plugin, allows learners to (1) set personalized learning goals, (2) engage in learning-by-doing through a conversational LLM-based mentor agent, (3) receive guidance and feedback based on a student model that steers the mentor moves. In a within-subject study with 16 participants learning exploratory data analysis from a streamcast, Tutorly significantly improved their performance from 61.9% to 76.6% based on a post-test questionnaire. Tutorly demonstrates the potential for enhancing programming video learning experiences with LLM and learner modeling.