VArsity: Can Large Language Models Keep Power Engineering Students in Phase?

TL;DR

This study explores the use of ChatGPT LLMs in power engineering education, highlighting challenges students face in identifying errors in different versions of the model and discussing implications for pedagogy.

Contribution

It provides an educational case study on deploying ChatGPT in power engineering courses, emphasizing the pedagogical impact and challenges of different LLM versions.

Findings

Students identified errors more easily with GPT-4 than with GPT-o1.

Error detection difficulty varied between LLM versions.

The role of LLMs in engineering education warrants further research.

Abstract

This paper provides an educational case study regarding our experience in deploying ChatGPT Large Language Models (LLMs) in the Spring 2025 and Fall 2023 offerings of ECE 4320: Power System Analysis and Control at Georgia Tech. As part of course assessments, students were tasked with identifying, explaining, and correcting errors in the ChatGPT outputs corresponding to power factor correction problems. While most students successfully identified the errors in the outputs from the GPT-4 version of ChatGPT used in Fall 2023, students found the errors from the ChatGPT o1 version much more difficult to identify in Spring 2025. As shown in this case study, the role of LLMs in pedagogy, assessment, and learning in power engineering classrooms is an important topic deserving further investigation.