Which Feedback Works for Whom? Differential Effects of LLM-Generated Feedback Elements Across Learner Profiles

TL;DR

This study investigates how different elements of LLM-generated feedback affect learning outcomes and acceptance among high school students with diverse personality traits, emphasizing personalized feedback design.

Contribution

It identifies how specific feedback elements influence learning and acceptance, and highlights the need for adapting feedback to individual personality profiles.

Findings

Certain feedback elements improve learning outcomes consistently.

Learners' preferences for feedback vary with personality traits.

Personalized feedback enhances learner acceptance and effectiveness.

Abstract

Large language models (LLMs) show promise for automatically generating feedback in education settings. However, it remains unclear how specific feedback elements, such as tone and information coverage, contribute to learning outcomes and learner acceptance, particularly across learners with different personality traits. In this study, we define six feedback elements and generate feedback for multiple-choice biology questions using GPT-5. We conduct a learning experiment with 321 first-year high school students and evaluate feedback effectiveness using two learning outcomes measures and subjective evaluations across six criteria. We further analyze differences in how feedback acceptance varies across learners based on Big Five personality traits. Our results show that effective feedback elements share common patterns supporting learning outcomes, while learners' subjective preferences differ across personality-based clusters. These findings highlight the importance of selecting and adapting feedback elements according to learners' personality traits when we design LLM-generated feedback, and provide practical implications for personalized feedback design in education.