An AI Teaching Assistant for Motion Picture Engineering
Deirdre O'Regan, Anil C. Kokaram

TL;DR
This study presents the implementation and evaluation of an AI Teaching Assistant using Retrieval Augmented Generation in a Motion Picture Engineering course, demonstrating its effectiveness and impact on student learning.
Contribution
It provides detailed implementation, tuning, and evaluation of an AI-TA in a university course, including its use in open-book exams and student feedback analysis.
Findings
AI-TA showed positive student feedback with a mean of 4.22/5.
No significant difference in exam performance with or without AI-TA access.
The experiment involved 43 students, 296 sessions, and 1,889 queries over 7 weeks.
Abstract
The rapid rise of LLMs over the last few years has promoted growing experimentation with LLM-driven AI tutors. However, the details of implementation, as well as the benefit in a teaching environment, are still in the early days of exploration. This article addresses these issues in the context of implementation of an AI Teaching Assistant (AI-TA) using Retrieval Augmented Generation (RAG) for Trinity College Dublin's Master's Motion Picture Engineering (MPE) course. We provide details of our implementation (including the prompt to the LLM, and code), and highlight how we designed and tuned our RAG pipeline to meet course needs. We describe our survey instrument and report on the impact of the AI-TA through a number of quantitative metrics. The scale of our experiment (43 students, 296 sessions, 1,889 queries over 7 weeks) was sufficient to have confidence in our findings. Unlike…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
