QACP: An Annotated Question Answering Dataset for Assisting Chinese Python Programming Learners

TL;DR

This paper introduces QACP, a Chinese question-answering dataset for Python learners, aiming to improve AI educational tools amid data scarcity, and evaluates LLMs' effectiveness in this context.

Contribution

The paper presents a new annotated Chinese Q&A dataset for Python learners, addressing data scarcity and enhancing AI-assisted programming education.

Findings

Questions sourced from real students ensure authenticity.

Evaluation reveals limitations of general LLMs as teaching assistants.

Dataset supports development of tailored educational LLMs.

Abstract

In online learning platforms, particularly in rapidly growing computer programming courses, addressing the thousands of students' learning queries requires considerable human cost. The creation of intelligent assistant large language models (LLMs) tailored for programming education necessitates distinct data support. However, in real application scenarios, the data resources for training such LLMs are relatively scarce. Therefore, to address the data scarcity in intelligent educational systems for programming, this paper proposes a new Chinese question-and-answer dataset for Python learners. To ensure the authenticity and reliability of the sources of the questions, we collected questions from actual student questions and categorized them according to various dimensions such as the type of questions and the type of learners. This annotation principle is designed to enhance the effectiveness and quality of online programming education, providing a solid data foundation for developing the programming teaching assists (TA). Furthermore, we conducted comprehensive evaluations of various LLMs proficient in processing and generating Chinese content, highlighting the potential limitations of general LLMs as intelligent teaching assistants in computer programming courses.