How Effective is GPT-4 Turbo in Generating School-Level Questions from Textbooks Based on Bloom's Revised Taxonomy?

TL;DR

This study assesses GPT-4 Turbo's ability to generate school-level questions from textbooks in zero-shot mode, focusing on Bloom's Revised Taxonomy levels, and compares its performance to human-generated questions.

Contribution

It provides an empirical evaluation of GPT-4 Turbo's effectiveness in generating higher-order thinking questions aligned with Bloom's taxonomy.

Findings

GPT-4 Turbo effectively generates questions requiring understanding and analysis.

There are differences between machine and human assessments of question quality.

Question quality evaluation varies with Bloom's taxonomy levels.

Abstract

We evaluate the effectiveness of GPT-4 Turbo in generating educational questions from NCERT textbooks in zero-shot mode. Our study highlights GPT-4 Turbo's ability to generate questions that require higher-order thinking skills, especially at the "understanding" level according to Bloom's Revised Taxonomy. While we find a notable consistency between questions generated by GPT-4 Turbo and those assessed by humans in terms of complexity, there are occasional differences. Our evaluation also uncovers variations in how humans and machines evaluate question quality, with a trend inversely related to Bloom's Revised Taxonomy levels. These findings suggest that while GPT-4 Turbo is a promising tool for educational question generation, its efficacy varies across different cognitive levels, indicating a need for further refinement to fully meet educational standards.