Improving Hybrid Human-AI Tutoring by Differentiating Human Tutor Roles Based on Student Needs

TL;DR

This study demonstrates that differentiated human-AI tutoring improves student engagement and achievement, with proactive support slightly more benefiting lower-performing students and narrowing achievement gaps.

Contribution

It introduces a differentiated tutoring policy based on student performance, showing its effectiveness in hybrid human-AI tutoring settings.

Findings

Significant improvements in time on task, skill proficiency, and academic growth from human-AI tutoring.

Proactive tutoring marginally outperforms reactive tutoring in MAP growth, especially for lower-performing students.

Differentiated tutoring effectively supports both high- and low-performing students, enhancing scalability.

Abstract

Hybrid human-AI tutoring, where technology and humans jointly facilitate student learning, can be more beneficial than AI-only tutoring. However, preliminary evidence suggests that lower-performing students derive greater benefit from human-AI tutoring than higher-performing students. As such, this study evaluates whether a differentiated tutoring policy can effectively support both groups: human tutors initiate support for lower-performing students, while higher-performing students receive reactive, on-demand support. Using their within-grade median state test scores, we assigned 635 students (grades 5-8) to receive proactive (< median) or reactive ($\geq$ median) tutoring. Using a DiDC design, we compare outcomes across two time periods: fall (AI-only tutoring) and spring (proactive-reactive human-AI tutoring). This quasi-experimental design isolates the effects of proactive-reactive tutoring approaches by comparing the discontinuity in spring outcomes to the fall, where no such discontinuity existed. Using data around the cutoff (Imbens-Kalyanaraman criterion), we find significant overall improvements from human-AI tutoring compared to AI-only baseline: 25% increase in time on task, 36% in skill proficiency, and 61% in academic growth (standardized MAP test). Between proactive and reactive tutoring, we find comparable improvements in time-on-task and skill proficiency. However, proactive tutoring, on average, showed marginally higher MAP growth (75%, p = .065) than reactive tutoring, i.e., proactive tutoring was more beneficial to students farther below the cutoff and helped narrow achievement gaps. Our findings provide evidence that differentiated human-AI tutoring addresses the needs of both groups, offering a practical and cost-effective strategy for scaling hybrid instruction.