From Co-Design to Metacognitive Laziness: Evaluating Generative AI in Vocational Education

TL;DR

This study evaluates a Generative AI tool in vocational education, highlighting its benefits for teachers but revealing complex impacts on student learning behaviors and equity, emphasizing the need for careful AI design.

Contribution

It provides insights into how Generative AI influences teaching workflows and student engagement, emphasizing the importance of addressing metacognitive and equity considerations.

Findings

Streamlined workflows and reduced cognitive load for teachers

No significant improvement in student assessment outcomes

Identification of dependency and metacognitive laziness patterns among students

Abstract

This study examines the development and deployment of a Generative AI proof-of-concept (POC) designed to support lecturers in a vocational education setting in Singapore. Employing a user-centred, mixed-methods design process, we co-developed an AI chatbot with lecturers to address recurring instructional challenges during exam preparation, specifically managing repetitive questions and scaling feedback delivery. The POC achieved its primary operational goals: lecturers reported streamlined workflows, reduced cognitive load, and observed improved student confidence in navigating course content. However, the deployment yielded unexpected insights into student learning behaviours. Despite enhanced teaching processes, performance data revealed no significant improvement in overall student assessment outcomes. Deep analysis of interaction logs identified concerning patterns, including self-efficacy-driven dependency, "metacognitive laziness" (cognitive offloading), and divergent usage strategies. While high-ability students leveraged the tool for strategic verification, low-ability students frequently used it to bypass cognitive effort, potentially exacerbating performance gaps. These findings suggest that Generative AI's educational influence extends beyond instructional efficiency to shape cognitive engagement, self-regulation, and learner equity. The study raises consequential design questions regarding how AI tools can be engineered to minimise dependency, scaffold metacognitive development, and calibrate support across varying ability levels. We conclude that while Generative AI can substantially enhance the teaching experience, achieving meaningful learning gains requires rigorous attention to learner behaviour and the equitable design of AI-supported environments.