Application of Large Language Models in Automated Question Generation: A Case Study on ChatGLM's Structured Questions for National Teacher Certification Exams

TL;DR

This study evaluates ChatGLM's ability to automatically generate structured questions for national teacher exams, showing high quality but also highlighting areas for improvement in question rating criteria.

Contribution

It demonstrates ChatGLM's effectiveness in generating exam questions with comparable quality to real questions, providing empirical evidence for AI-assisted educational assessment.

Findings

ChatGLM-generated questions are similar in quality to real exam questions.

Expert evaluations confirm the rationality and practicality of generated questions.

Limitations exist in the model's consideration of diverse rating criteria.

Abstract

This study delves into the application potential of the large language models (LLMs) ChatGLM in the automatic generation of structured questions for National Teacher Certification Exams (NTCE). Through meticulously designed prompt engineering, we guided ChatGLM to generate a series of simulated questions and conducted a comprehensive comparison with questions recollected from past examinees. To ensure the objectivity and professionalism of the evaluation, we invited experts in the field of education to assess these questions and their scoring criteria. The research results indicate that the questions generated by ChatGLM exhibit a high level of rationality, scientificity, and practicality similar to those of the real exam questions across most evaluation criteria, demonstrating the model's accuracy and reliability in question generation. Nevertheless, the study also reveals limitations in the model's consideration of various rating criteria when generating questions, suggesting the need for further optimization and adjustment. This research not only validates the application potential of ChatGLM in the field of educational assessment but also provides crucial empirical support for the development of more efficient and intelligent educational automated generation systems in the future.