Brief but Impactful: How Human Tutoring Interactions Shape Engagement in Online Learning

TL;DR

This study analyzes how brief, well-timed human tutoring interactions during online math practice significantly boost student engagement, with implications for optimizing resource allocation in educational settings.

Contribution

It introduces a data-driven approach to identify effective tutor-student dialogues and highlights the importance of timing and interaction style in enhancing engagement.

Findings

Engagement increases during and after tutor visits.

Later visits produce larger immediate engagement lifts.

Concrete, stepwise scaffolding with explicit work organization most effectively boosts engagement.

Abstract

Learning analytics can guide human tutors to efficiently address motivational barriers to learning that AI systems struggle to support. Students become more engaged when they receive human attention. However, what occurs during short interventions, and when are they most effective? We align student-tutor dialogue transcripts with MATHia tutoring system log data to study brief human-tutor interactions on Zoom drawn from 2,075 hours of 191 middle school students' classroom math practice. Mixed-effect models reveal that engagement, measured as successful solution steps per minute, is higher during a human-tutor visit and remains elevated afterward. Visit length exhibits diminishing returns: engagement rises during and shortly after visits, irrespective of visit length. Timing also matters: later visits yield larger immediate lifts than earlier ones, though an early visit remains important to counteract engagement decline. We create analytics that identify which tutor-student dialogues raise engagement the most. Qualitative analysis reveals that interactions with concrete, stepwise scaffolding with explicit work organization elevate engagement most strongly. We discuss implications for resource-constrained tutoring, prioritizing several brief, well-timed check-ins by a human tutor while ensuring at least one early contact. Our analytics can guide the prioritization of students for support and surface effective tutor moves in real-time.