AI Chatbots as Multi-Role Pedagogical Agents: Transforming Engagement in CS Education

TL;DR

This paper explores the development and evaluation of AI-powered multi-role chatbots in CS education, demonstrating their potential to enhance engagement, motivation, and inquiry-based learning through a novel, psychologically grounded system tested with 200 students.

Contribution

It introduces a multi-role chatbot system based on Self-Determination Theory, integrating NLP techniques and mixed-methods evaluation in higher education.

Findings

Increased learner engagement and motivation observed.

Enhanced inquiry and exploration among students.

Positive feedback on chatbot roles and system design.

Abstract

This study investigates the use of Artificial Intelligence (AI)-powered, multi-role chatbots as a means to enhance learning experiences and foster engagement in computer science education. Leveraging a design-based research approach, we develop, implement, and evaluate a novel learning environment enriched with four distinct chatbot roles: Instructor Bot, Peer Bot, Career Advising Bot, and Emotional Supporter Bot. These roles, designed around the tenets of Self-Determination Theory, cater to the three innate psychological needs of learners - competence, autonomy, and relatedness. Additionally, the system embraces an inquiry-based learning paradigm, encouraging students to ask questions, seek solutions, and explore their curiosities. We test this system in a higher education context over a period of one month with 200 participating students, comparing outcomes with conditions involving a human tutor and a single chatbot. Our research utilizes a mixed-methods approach, encompassing quantitative measures such as chat log sequence analysis, and qualitative methods including surveys and focus group interviews. By integrating cutting-edge Natural Language Processing techniques such as topic modelling and sentiment analysis, we offer an in-depth understanding of the system's impact on learner engagement, motivation, and inquiry-based learning. This study, through its rigorous design and innovative approach, provides significant insights into the potential of AI-empowered, multi-role chatbots in reshaping the landscape of computer science education and fostering an engaging, supportive, and motivating learning environment.