Exploring Teacher-Chatbot Interaction and Affect in Block-Based Programming

TL;DR

This study explores middle school teachers' interactions with LLM-based chatbots in block-based programming, highlighting diverse needs, perceived benefits, risks, and design recommendations to improve educational chatbot integration.

Contribution

It provides a qualitative analysis of teachers' perspectives, proposing profiles and design guidelines for effective chatbot integration in middle school coding education.

Findings

Teachers see benefits in prompting skills and confidence building.

Risks include potential declines in learning and critical thinking.

Design recommendations focus on scaffolding and teacher control.

Abstract

AI-based chatbots have the potential to accelerate learning and teaching, but may also have counterproductive consequences without thoughtful design and scaffolding. To better understand teachers' perspectives on large language model (LLM)-based chatbots, we conducted a study with 11 teams of middle school teachers using chatbots for a science and computational thinking activity within a block-based programming environment. Based on a qualitative analysis of audio transcripts and chatbot interactions, we propose three profiles: explorer, frustrated, and mixed, that reflect diverse scaffolding needs. In their discussions, we found that teachers perceived chatbot benefits such as building prompting skills and self-confidence alongside risks including potential declines in learning and critical thinking. Key design recommendations include scaffolding the introduction to chatbots, facilitating teacher control of chatbot features, and suggesting when and how chatbots should be used. Our contribution informs the design of chatbots to support teachers and learners in middle school coding activities.