Evaluating the capability of large language models to personalize science texts for diverse middle-school-age learners

TL;DR

This study demonstrates that GPT-4 can personalize science texts for middle school students based on their learning preferences, leading to increased student preference for tailored content, thus advancing personalized learning technology.

Contribution

It is among the first to empirically evaluate GPT-4's ability to personalize educational science texts for diverse middle school learners.

Findings

Students preferred texts aligned with their learning profiles (p = .059).

GPT-4 effectively interprets and tailors content to individual preferences.

Study highlights potential of LLMs in personalized education.

Abstract

Large language models (LLMs), including OpenAI's GPT-series, have made significant advancements in recent years. Known for their expertise across diverse subject areas and quick adaptability to user-provided prompts, LLMs hold unique potential as Personalized Learning (PL) tools. Despite this potential, their application in K-12 education remains largely unexplored. This paper presents one of the first randomized controlled trials (n = 23) to evaluate the effectiveness of GPT-4 in personalizing educational science texts for middle school students. In this study, GPT-4 was used to profile student learning preferences based on choices made during a training session. For the experimental group, GPT-4 was used to rewrite science texts to align with the student's predicted profile while, for students in the control group, texts were rewritten to contradict their learning preferences. The results of a Mann-Whitney U test showed that students significantly preferred (at the .10 level) the rewritten texts when they were aligned with their profile (p = .059). These findings suggest that GPT-4 can effectively interpret and tailor educational content to diverse learner preferences, marking a significant advancement in PL technology. The limitations of this study and ethical considerations for using artificial intelligence in education are also discussed.