CLASS: A Design Framework for building Intelligent Tutoring Systems based on Learning Science principles

TL;DR

This paper introduces the CLASS framework for developing advanced Intelligent Tutoring Systems that leverage large language models to provide step-by-step guidance and natural language interaction, demonstrated through a biology tutor called SPOCK.

Contribution

The paper presents a novel design framework, CLASS, that integrates problem-solving strategies and conversational capabilities into ITS using LLMs, with a proof-of-concept implementation for biology education.

Findings

SPOCK effectively breaks down complex biology questions into subproblems.

SPOCK provides accurate and relevant responses in tutoring scenarios.

Experts favor SPOCK's ability to engage students and facilitate learning.

Abstract

We present a design framework called Conversational Learning with Analytical Step-by-Step Strategies (CLASS) for building advanced Intelligent Tutoring Systems (ITS) powered by high-performance Large Language Models (LLMs). The CLASS framework empowers ITS with two key capabilities. First, through a carefully curated scaffolding dataset, CLASS equips ITS with essential problem-solving strategies, enabling it to provide tutor-like, step-by-step guidance to students. Second, by using a dynamic conversational dataset, CLASS assists ITS in facilitating natural language interactions, fostering engaging student-tutor conversations. The CLASS framework also provides valuable insights into ITS' internal decision-making process which allows seamless integration of user feedback, thus enabling continuous refinement and improvement. We also present a proof-of-concept ITS, referred to as SPOCK, which is trained using the CLASS framework with a focus on introductory college-level biology content. A carefully constructed protocol was developed for SPOCK's preliminary evaluation, examining aspects such as the factual accuracy and relevance of its responses. Experts in the field of biology offered favorable remarks, particularly highlighting SPOCK's capability to break down questions into manageable subproblems and provide encouraging responses to students. Code and models are available at https://github.com/luffycodes/Tutorbot-Spock.