VTutor: An Animated Pedagogical Agent SDK that Provide Real Time Multi-Model Feedback

TL;DR

VTutor is an open-source SDK that creates real-time, animated pedagogical agents with personalized feedback, enhancing student engagement and learning through expressive, stylized characters and improved synchronization.

Contribution

Introduces VTutor, a lightweight, web-based SDK for real-time animated pedagogical agents that overcome previous limitations with natural animations and low development costs.

Findings

VTutor significantly outperforms existing approaches in perceived synchronization and naturalness.

Participants preferred VTutor's expressive and stylized characters over traditional talking-heads.

The system enables scalable, personalized educational interactions with high user engagement.

Abstract

Pedagogical Agents (PAs) show significant potential for boosting student engagement and learning outcomes by providing adaptive, on-demand support in educational contexts. However, existing PA solutions are often hampered by pre-scripted dialogue, unnatural animations, uncanny visual realism, and high development costs. To address these gaps, we introduce VTutor, an open-source SDK leveraging lightweight WebGL, Unity, and JavaScript frameworks. VTutor receives text outputs from a large language model (LLM), converts them into audio via text-to-speech, and then renders a real-time, lip-synced pedagogical agent (PA) for immediate, large-scale deployment on web-based learning platforms. By providing on-demand, personalized feedback, VTutor strengthens students' motivation and deepens their engagement with instructional material. Using an anime-like aesthetic, VTutor alleviates the uncanny valley effect, allowing learners to engage with expressive yet comfortably stylized characters. Our evaluation with 50 participants revealed that VTutor significantly outperforms the existing talking-head approaches (e.g., SadTalker) on perceived synchronization accuracy, naturalness, emotional expressiveness, and overall preference. As an open-source project, VTutor welcomes community-driven contributions - from novel character designs to specialized showcases of pedagogical agent applications - that fuel ongoing innovation in AI-enhanced education. By providing an accessible, customizable, and learner-centered PA solution, VTutor aims to elevate human-AI interaction experience in education fields, ultimately broadening the impact of AI in learning contexts. The demo link to VTutor is at https://vtutor-aied25.vercel.app.