Towards Pedagogical LLMs with Supervised Fine Tuning for Computing Education

TL;DR

This paper explores supervised fine-tuning of large language models using a proprietary dataset to enhance their pedagogical alignment in computing education, aiming to improve learning outcomes and adherence to educational principles.

Contribution

It introduces a novel approach of using university forum data for fine-tuning LLMs to better align with pedagogical principles in computing education.

Findings

Improved pedagogical alignment observed in fine-tuned LLMs

University course forums are suitable for creating educational datasets

Further evaluations needed for comprehensive assessment

Abstract

This paper investigates supervised fine-tuning of large language models (LLMs) to improve their pedagogical alignment in computing education, addressing concerns that LLMs may hinder learning outcomes. The project utilised a proprietary dataset of 2,500 high quality question/answer pairs from programming course forums, and explores two research questions: the suitability of university course forums in contributing to fine-tuning datasets, and how supervised fine-tuning can improve LLMs' alignment with educational principles such as constructivism. Initial findings suggest benefits in pedagogical alignment of LLMs, with deeper evaluations required.