When learning analytics dashboard is explainable: An exploratory study on the effect of GenAI-supported learning analytics dashboard

TL;DR

This study explores how an explainable learning analytics dashboard, grounded in SRL and XAI principles, influences students' understanding and collaboration in academic writing with AI, showing deeper learning benefits.

Contribution

It introduces a theory-driven, explainable LAD design and empirically evaluates its impact on students' conceptual understanding in AI-supported academic writing.

Findings

Students using the explainable LAD showed better grasp of writing principles.

No significant difference in abstract quality between groups.

Explainable feedback enhances deeper learning and skill transfer.

Abstract

This study investigated the impact of a theory-driven, explainable Learning Analytics Dashboard (LAD) on university students' human-AI collaborative academic abstract writing task. Grounded in Self-Regulated Learning (SRL) theory and incorporating Explainable AI (XAI) principles, our LAD featured a three-layered design (Visual, Explainable, Interactive). In an experimental study, participants were randomly assigned to either an experimental group (using the full explainable LAD) or a control group (using a visual-only LAD) to collaboratively write an academic abstract with a Generative AI. While quantitative analysis revealed no significant difference in the quality of co-authored abstracts between the two groups, a significant and noteworthy difference emerged in conceptual understanding: students in the explainable LAD group demonstrated a superior grasp of abstract writing principles, as evidenced by their higher scores on a knowledge test (p= .026). These findings highlight that while basic AI-generated feedback may suffice for immediate task completion, the provision of explainable feedback is crucial for fostering deeper learning, enhancing conceptual understanding, and developing transferable skills fundamental to self-regulated learning in academic writing contexts.