Small but Significant: On the Promise of Small Language Models for Accessible AIED

TL;DR

This paper highlights the potential of small language models (SLMs) like Phi-2 to provide accessible, affordable AI solutions in education, especially for resource-constrained institutions, challenging the focus on large models like GPT.

Contribution

It demonstrates that SLMs can effectively address key challenges in AIED, such as knowledge component discovery, without the need for complex prompting strategies.

Findings

SLMs like Phi-2 perform well in knowledge component discovery.

SLMs offer a resource-efficient alternative to large language models.

The paper advocates for increased focus on SLM development in AIED.

Abstract

GPT has become nearly synonymous with large language models (LLMs), an increasingly popular term in AIED proceedings. A simple keyword-based search reveals that 61% of the 76 long and short papers presented at AIED 2024 describe novel solutions using LLMs to address some of the long-standing challenges in education, and 43% specifically mention GPT. Although LLMs pioneered by GPT create exciting opportunities to strengthen the impact of AI on education, we argue that the field's predominant focus on GPT and other resource-intensive LLMs (with more than 10B parameters) risks neglecting the potential impact that small language models (SLMs) can make in providing resource-constrained institutions with equitable and affordable access to high-quality AI tools. Supported by positive results on knowledge component (KC) discovery, a critical challenge in AIED, we demonstrate that SLMs such as Phi-2 can produce an effective solution without elaborate prompting strategies. Hence, we call for more attention to developing SLM-based AIED approaches.