The Effects of Generative AI Agents and Scaffolding on Enhancing Students' Comprehension of Visual Learning Analytics

TL;DR

This study explores how generative AI agents, especially proactive ones, combined with scaffolding, can significantly improve students' understanding of visual learning analytics, outperforming traditional methods.

Contribution

It demonstrates that proactive GenAI agents with scaffolding techniques substantially enhance VLA comprehension and have lasting educational benefits.

Findings

Proactive GenAI agents outperform passive agents and standalone scaffolding.

Passive GenAI agents show similar improvements to traditional scaffolding.

Benefits of proactive agents persist beyond the intervention period.

Abstract

Visual learning analytics (VLA) is becoming increasingly adopted in educational technologies and learning analytics dashboards to convey critical insights to students and educators. Yet many students experienced difficulties in comprehending complex VLA due to their limited data visualisation literacy. While conventional scaffolding approaches like data storytelling have shown effectiveness in enhancing students' comprehension of VLA, these approaches remain difficult to scale and adapt to individual learning needs. Generative AI (GenAI) technologies, especially conversational agents, offer potential solutions by providing personalised and dynamic support to enhance students' comprehension of VLA. This study investigates the effectiveness of GenAI agents, particularly when integrated with scaffolding techniques, in improving students' comprehension of VLA. A randomised controlled trial was conducted with 117 higher education students to compare the effects of two types of GenAI agents: passive agents, which respond to student queries, and proactive agents, which utilise scaffolding questions, against standalone scaffolding in a VLA comprehension task. The results show that passive agents yield comparable improvements to standalone scaffolding both during and after the intervention. Notably, proactive GenAI agents significantly enhance students' VLA comprehension compared to both passive agents and standalone scaffolding, with these benefits persisting beyond the intervention. These findings suggest that integrating GenAI agents with scaffolding can have lasting positive effects on students' comprehension skills and support genuine learning.