Impact of Criterion-Based Reflection on Prospective Physics Teachers' Perceptions of ChatGPT-Generated Content

TL;DR

This study investigates how criterion-based reflection influences prospective physics teachers' perceptions of ChatGPT-generated content, emphasizing the importance of critical thinking and media literacy in teacher training.

Contribution

It introduces a criterion-based evaluation approach to assess how critical reflection affects perceptions of ChatGPT's answer quality among prospective teachers.

Findings

Critical reflection shifts perceptions of answer quality.

Increased awareness of ChatGPT's limitations.

Emergence of extreme positive, negative, and balanced perspectives.

Abstract

ChatGPT has significantly shaped digital transformation discussions. Its widespread testing and ongoing optimization highlight the need to assess its capabilities and encourage critical reflection. This study explores how students' critical reflection on ChatGPT-generated content impacts their perceptions of its answer quality and further use. Involving 39 prospective physics teachers, the study assessed their evaluations of ChatGPT's answers to didactical tasks using predefined criteria, in this paper referred as the criterion-based evaluation approach. Pre- and post-questionnaires, with a 5-point ranking scale and open-ended questions, evaluated students' perceptions of ChatGPT's helpfulness and quality. Results showed that critical reflection shifted students' perception of answer quality and increased their awareness of ChatGPT's limitations. Three perspectives on ChatGPT's further use emerged: extreme positive, extreme negative, and balanced. The extreme positive perspective indicated confirmation bias, where the positive prior experiences of students with ChatGPT influenced their evaluations. The findings highlight the need to foster critical thinking and media literacy in teacher training programs to help educators effectively integrate such tools.