Battling Botpoop using GenAI for Higher Education: A Study of a Retrieval Augmented Generation Chatbots Impact on Learning

TL;DR

This study presents Professor Leodar, a custom retrieval-augmented GenAI chatbot in Singaporean higher education, demonstrating positive impacts on student engagement and learning, while addressing issues of low-quality information in AI-assisted education.

Contribution

Introduces a novel Singlish-speaking RAG chatbot for education, deploying it in a real university setting and evaluating its effects on learning and engagement.

Findings

97.1% participants reported positive experiences

Enhanced personalized guidance and 24/7 availability

Benchmark for future AI educational tools

Abstract

Generative artificial intelligence (GenAI) and large language models (LLMs) have simultaneously opened new avenues for enhancing human learning and increased the prevalence of poor-quality information in student response - termed Botpoop. This study introduces Professor Leodar, a custom-built, Singlish-speaking Retrieval Augmented Generation (RAG) chatbot designed to enhance educational while reducing Botpoop. Deployed at Nanyang Technological University, Singapore, Professor Leodar offers a glimpse into the future of AI-assisted learning, offering personalized guidance, 24/7 availability, and contextually relevant information. Through a mixed-methods approach, we examine the impact of Professor Leodar on learning, engagement, and exam preparedness, with 97.1% of participants reporting positive experiences. These findings help define possible roles of AI in education and highlight the potential of custom GenAI chatbots. Our combination of chatbot development, in-class deployment and outcomes study offers a benchmark for GenAI educational tools and is a stepping stone for redefining the interplay between AI and human learning.