How undergraduate physics students use generative AI for computational modeling

TL;DR

This study explores how undergraduate physics students utilize generative AI in computational modeling, revealing its benefits, challenges, and implications for teaching practices in physics education.

Contribution

It provides empirical insights into students' use of genAI in physics modeling and discusses pedagogical strategies to enhance productive engagement with AI tools.

Findings

GenAI aids in planning, implementing, and debugging models.

Students often over-rely on genAI, leading to misconceptions.

Teaching should focus on productive use and fundamental understanding.

Abstract

Generative artificial intelligence (genAI) is becoming increasingly prevalent and capable in physics, particularly for programming-related tasks. How, then, does genAI affect students' computational modeling? We interviewed 19 undergraduate students who had recently completed an open-ended computational assignment that encouraged the use of genAI, asking them how they used it. We then conducted a thematic analysis of these interviews using a framework for computational modeling in physics. We found that genAI significantly impacts several aspects of students' computational modeling, such as the planning, implementing, and debugging of computational models. GenAI can also help students find resources and introduce them to new computational tools. Productive use of genAI was associated with students limiting its use to small steps in the modeling process and consistently double-checking the formulas, explanations, and code it provided. We also identified challenges students faced due to an over-reliance on genAI, such as working from false model assumptions and not spending time learning the fundamentals of computational modeling, especially debugging. Finally, we discuss implications for teaching, such as the need to teach students how to use genAI productively and to urge them to plan before they code. We also highlight the continued value of low-stakes assessment and teaching assistants for teaching computational modeling, as the task remains difficult even with the introduction of genAI.