Revolutionizing Undergraduate Learning: CourseGPT and Its Generative AI Advancements

TL;DR

This paper presents CourseGPT, an AI tool built on open-source LLMs to support undergraduate education by providing dynamic, course-specific responses, improving learning experiences, and streamlining administrative tasks.

Contribution

The paper introduces CourseGPT, a novel generative AI system tailored for educational support, utilizing open-source LLMs and enabling instructor control over responses.

Findings

Higher parameter models like Mixtral-8x7b outperform smaller models.

CourseGPT achieves 88.0% correctness and 66.6% faithfulness scores.

Students and TAs find CourseGPT accurate and helpful.

Abstract

Integrating Generative AI (GenAI) into educational contexts presents a transformative potential for enhancing learning experiences. This paper introduces CourseGPT, a generative AI tool designed to support instructors and enhance the educational experiences of undergraduate students. Built on open-source Large Language Models (LLMs) from Mistral AI, CourseGPT offers continuous instructor support and regular updates to course materials, enriching the learning environment. By utilizing course-specific content, such as slide decks and supplementary readings and references, CourseGPT provides precise, dynamically generated responses to student inquiries. Unlike generic AI models, CourseGPT allows instructors to manage and control the responses, thus extending the course scope without overwhelming details. The paper demonstrates the application of CourseGPT using the CPR E 431 - Basics of Information System Security course as a pilot. This course, with its large enrollments and diverse curriculum, serves as an ideal testbed for CourseGPT. The tool aims to enhance the learning experience, accelerate feedback processes, and streamline administrative tasks. The study evaluates CourseGPT's impact on student outcomes, focusing on correctness scores, context recall, and faithfulness of responses. Results indicate that the Mixtral-8x7b model, with a higher parameter count, outperforms smaller models, achieving an 88.0% correctness score and a 66.6% faithfulness score. Additionally, feedback from former students and teaching assistants on CourseGPT's accuracy, helpfulness, and overall performance was collected. The outcomes revealed that a significant majority found CourseGPT to be highly accurate and beneficial in addressing their queries, with many praising its ability to provide timely and relevant information.