Automated Bias Assessment in AI-Generated Educational Content Using CEAT Framework

TL;DR

This paper introduces an automated method for detecting biases in AI-generated educational content, combining the CEAT framework with retrieval-augmented techniques to improve fairness and scalability.

Contribution

It presents a novel automated bias assessment approach specifically designed for educational materials generated by GenAI, integrating CEAT with prompt-engineered extraction within a RAG framework.

Findings

High correlation (r=0.993) between automated and manual bias assessments

Method reduces human bias and increases scalability in bias detection

Enhances fairness and reproducibility in educational content auditing

Abstract

Recent advances in Generative Artificial Intelligence (GenAI) have transformed educational content creation, particularly in developing tutor training materials. However, biases embedded in AI-generated content--such as gender, racial, or national stereotypes--raise significant ethical and educational concerns. Despite the growing use of GenAI, systematic methods for detecting and evaluating such biases in educational materials remain limited. This study proposes an automated bias assessment approach that integrates the Contextualized Embedding Association Test with a prompt-engineered word extraction method within a Retrieval-Augmented Generation framework. We applied this method to AI-generated texts used in tutor training lessons. Results show a high alignment between the automated and manually curated word sets, with a Pearson correlation coefficient of r = 0.993, indicating reliable and consistent bias assessment. Our method reduces human subjectivity and enhances fairness, scalability, and reproducibility in auditing GenAI-produced educational content.