Evaluation of ChatGPT and Microsoft Bing AI Chat Performances on Physics Exams of Vietnamese National High School Graduation Examination

TL;DR

This study evaluates ChatGPT and BingChat's performance on Vietnamese high school physics exams, revealing they underperform compared to students and are limited in high-level application questions, but can assist in education.

Contribution

First comprehensive assessment of ChatGPT and BingChat on Vietnamese physics exams, highlighting their current limitations and potential educational benefits.

Findings

Both LLMs perform worse than Vietnamese students.

Neither LLM can answer high application level questions.

BingChat generally more accurate, ChatGPT more stable.

Abstract

The promise and difficulties of language model-based approaches for physics teaching were assessed in this study. This study evaluates how well ChatGPT and BingChat, two state-of-the-art (SOTA) large language models (LLMs), perform when answering high school physics questions on Vietnamese exams from 2019 to 2023. When we compared the results of the LLMs with the scores of Vietnamese students, we discovered that ChatGPT and BingChat both perform worse than Vietnamese students, proving that LLMs are not yet capable of fully replacing human intellect in the field of physics teaching. The outcomes also showed that neither LLM is capable of responding to questions at the high application levels. In terms of accuracy, BingChat typically surpassed ChatGPT, although ChatGPT showed more stability. Our research suggests that LLMs can help students and teachers during learning and teaching activities, particularly by offering immediate feedback and individualized learning experiences.