Generative AI Feedback, English Writing and Teacher Rubrics: A Multiple-Case Study of CyberScholar

TL;DR

This study explores how a Generative AI tool, CyberScholar, supports K-12 writing by providing rubric-based feedback, improving student revision, and saving teacher time, with considerations for its limitations.

Contribution

It demonstrates the potential of rubric-grounded GenAI for formative feedback in K-12 writing and highlights practical insights for implementation and improvement.

Findings

Students valued immediate, rubric-based feedback and improved their writing through revision.

CyberScholar reduced teacher feedback time and supported targeted instruction.

Participants noted inconsistencies in automated ratings and alignment issues.

Abstract

This multiple-case study examined the potential of a Generative AI (GenAI) tool, CyberScholar, to support K-12 students' writing across disciplines. This tool integrates teacher-provided rubrics, materials, and exemplars through Retrieval-Augmented Generation (RAG), producing criterion-specific formative feedback and ratings. The study involved 143 students and five teachers in grades 7 through 11 across five U.S. middle and high schools. Data sources included classroom observations, student post-surveys (n = 79), student focus group interviews (n = 18), and teacher surveys (n = 5). Qualitative analysis followed two cycles of coding to identify patterns within and across cases. Findings indicate that students valued CyberScholar's immediate, rubric-based feedback and noticed improvements in their writing as they revised, using it to refine organization, elaboration, and style. They also highlighted the tool's interactive, iterative qualities, which fostered revision and reduced reliance on teacher feedback. However, participants noted inconsistencies in the automated rating system and occasional misalignment with assignment expectations. Teachers reported that CyberScholar saved time on feedback and supported more targeted, higher-order instructional practices. The study underscores the promise of rubric-grounded GenAI formative feedback for developing writing skills, while emphasizing the need for human oversight, calibration of automated ratings, and attention to contextual factors shaping adoption.