Student and AI responses to physics problems examined through the lenses of sensemaking and mechanistic reasoning

TL;DR

This study compares student and AI-generated responses to physics problems, revealing differences in how physics is talked about versus practiced, with implications for AI integration in physics education.

Contribution

It introduces a mixed-methods approach to analyze and compare student and AI responses to physics problems through sensemaking and mechanistic reasoning.

Findings

AI responses are well-structured and reflect physics talk.

Student responses effectively leverage representations and refine arguments.

AI responses mirror how physics is discussed, students how it is practiced.

Abstract

Several reports in education have called for transforming physics learning environments by promoting sensemaking of real-world scenarios in light of curricular ideas. Recent advancements in Generative-Artificial Intelligence has garnered increasing traction in educators' community by virtue of its potential in transforming STEM learning. In this exploratory study, we adopt a mixed-methods approach in comparatively examining student- and AI-generated responses to two different formats of a physics problem through the cognitive lenses of sensemaking and mechanistic reasoning. The student data is derived from think-aloud interviews of introductory students and the AI data comes from ChatGPT's solutions collected using Zero shot approach. The results highlight AI responses to evidence most features of the two processes through well-structured solutions and student responses to effectively leverage representations in their solutions through iterative refinement of arguments. In other words, while AI responses reflect how physics is talked about, the student responses reflect how physics is practiced. Implications of these results in light of development and deployment of AI systems in physics pedagogy are discussed.