Assessing Changes in Thinking about Troubleshooting in Physical Computing: A Clinical Interview Protocol with Failure Artifacts Scenarios

TL;DR

This paper introduces a clinical interview protocol with failure artifact scenarios to assess how high school students' troubleshooting explanations in physical computing evolve over an eight-week course, capturing nuanced thinking changes.

Contribution

It presents a novel assessment method that reveals students' troubleshooting reasoning processes beyond traditional correctness tests.

Findings

Protocol effectively captures changes in troubleshooting explanations.

Students demonstrate increased understanding of domain-specific issues.

The method can evaluate development of troubleshooting skills over time.

Abstract

Purpose: The purpose of this paper is to examine how a clinical interview protocol with failure artifact scenarios can capture changes in high school students' explanations of troubleshooting processes in physical computing activities. We focus on physical computing since finding and fixing hardware and software bugs is a highly contextual practice that involves multiple interconnected domains and skills. Approach: We developed and piloted a "failure artifact scenarios" clinical interview protocol. Youth were presented with buggy physical computing projects over video calls and asked for suggestions on how to fix them without having access to the actual project or its code. We applied this clinical interview protocol before and after an eight-week-long physical computing (more specifically, electronic textiles) unit. We analyzed matching pre- and post-interviews from 18 students at four different schools. Findings: Our findings demonstrate how the protocol can capture change in students' thinking about troubleshooting by eliciting students' explanations of specificity of domain knowledge of problems, multimodality of physical computing, iterative testing of failure artifact scenarios, and concreteness of troubleshooting and problem solving processes. Originality: Beyond tests and surveys used to assess debugging, which traditionally focus on correctness or student beliefs, our "failure artifact scenarios" clinical interview protocol reveals student troubleshooting-related thinking processes when encountering buggy projects. As an assessment tool, it may be useful to evaluate the change and development of students' abilities over time.