TL;DR
This study evaluates how generative AI, specifically large language models, can assess and improve tutors' equity-related skills in online education, showing promising results in performance and confidence gains.
Contribution
It introduces a novel application of LLMs for assessing tutor responses to equity scenarios and provides a dataset for future research in scalable equity training assessment.
Findings
Marginally significant increase in tutor confidence post-training
GPT-4o with few-shot learning effectively assesses tutor performance
Dataset and prompts made publicly available for further research
Abstract
Equity is a core concern of learning analytics. However, applications that teach and assess equity skills, particularly at scale are lacking, often due to barriers in evaluating language. Advances in generative AI via large language models (LLMs) are being used in a wide range of applications, with this present work assessing its use in the equity domain. We evaluate tutor performance within an online lesson on enhancing tutors' skills when responding to students in potentially inequitable situations. We apply a mixed-method approach to analyze the performance of 81 undergraduate remote tutors. We find marginally significant learning gains with increases in tutors' self-reported confidence in their knowledge in responding to middle school students experiencing possible inequities from pretest to posttest. Both GPT-4o and GPT-4-turbo demonstrate proficiency in assessing tutors ability to…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
