Investigating Conversational Agents to Support Secondary School Students Learning CSP

TL;DR

This study evaluates how conversational agents, including ChatGPT and custom tools, can support secondary school students learning AP CSP concepts, focusing on effectiveness and engagement.

Contribution

It provides insights into the potential of general-purpose and custom conversational agents to assist CSP students and evaluates their effectiveness in classroom settings.

Findings

Conversational agents can effectively support CSP learning.

Custom fixed-response agents increased student engagement.

ChatGPT showed promise as an exploratory learning tool.

Abstract

Secondary school students enrolled in the AP Computer Science Principles (CSP) course commonly utilize web resources (e.g., tutorials, Q\&A sites) to better understand key concepts in the curriculum. The primary obstacle to using these resources is finding information appropriate for the learning task and student's background. In addition to web search, conversational agents are increasingly a viable alternative for CSP students. In this paper, we study the potential of conversational agents to aid secondary school students as they acquire knowledge on CSP concepts. We explore general purpose, generative conversational agents (e.g., ChatGPT) and custom, fixed-response conversational agents built specifically to aid CSP students. We present results from classroom use by 45 high school students in grades 9-11 (ages 14-17) across six CSP sections. Our main contributions are in better understanding how conversational agents can help CSP students and an evaluation of the effectiveness and engagement of different approaches for CSP exploratory search.