Educators' Perceptions of Large Language Models as Tutors: Comparing Human and AI Tutors in a Blind Text-only Setting

TL;DR

This study compares human and AI tutors using LLMs in a blind setting, finding that LLMs are perceived as more engaging, empathetic, and effective in scaffolding and conciseness, suggesting potential to support teachers.

Contribution

It provides a comparative analysis of human versus LLM tutors on key teaching qualities, highlighting LLMs' strengths in empathy and engagement.

Findings

LLMs are perceived as more empathetic by 80% of annotators.

Annotators with teaching experience rate LLMs higher in all four metrics.

Results suggest LLMs can complement human tutors to reduce teacher workload.

Abstract

The rapid development of Large Language Models (LLMs) opens up the possibility of using them as personal tutors. This has led to the development of several intelligent tutoring systems and learning assistants that use LLMs as back-ends with various degrees of engineering. In this study, we seek to compare human tutors with LLM tutors in terms of engagement, empathy, scaffolding, and conciseness. We ask human tutors to annotate and compare the performance of an LLM tutor with that of a human tutor in teaching grade-school math word problems on these qualities. We find that annotators with teaching experience perceive LLMs as showing higher performance than human tutors in all 4 metrics. The biggest advantage is in empathy, where 80% of our annotators prefer the LLM tutor more often than the human tutors. Our study paints a positive picture of LLMs as tutors and indicates that these models can be used to reduce the load on human teachers in the future.