Automated Analysis of Learning Outcomes and Exam Questions Based on Bloom's Taxonomy

TL;DR

This study compares traditional machine learning, neural networks, transformer models, and large language models for classifying exam questions and learning outcomes according to Bloom's Taxonomy, highlighting the effectiveness of data augmentation and simpler models.

Contribution

It provides a comprehensive evaluation of various models and strategies for automating Bloom's Taxonomy classification, emphasizing the strengths of SVM with data augmentation on limited datasets.

Findings

SVM with data augmentation achieved 94% accuracy.

Deep models like BERT and RNNs overfit on small datasets.

Large language models performed best in zero-shot evaluations.

Abstract

This paper explores the automatic classification of exam questions and learning outcomes according to Bloom's Taxonomy. A small dataset of 600 sentences labeled with six cognitive categories - Knowledge, Comprehension, Application, Analysis, Synthesis, and Evaluation - was processed using traditional machine learning (ML) models (Naive Bayes, Logistic Regression, Support Vector Machines), recurrent neural network architectures (LSTM, BiLSTM, GRU, BiGRU), transformer-based models (BERT and RoBERTa), and large language models (OpenAI, Gemini, Ollama, Anthropic). Each model was evaluated under different preprocessing and augmentation strategies (for example, synonym replacement, word embeddings, etc.). Among traditional ML approaches, Support Vector Machines (SVM) with data augmentation achieved the best overall performance, reaching 94 percent accuracy, recall, and F1 scores with minimal overfitting. In contrast, the RNN models and BERT suffered from severe overfitting, while RoBERTa initially overcame it but began to show signs as training progressed. Finally, zero-shot evaluations of large language models (LLMs) indicated that OpenAI and Gemini performed best among the tested LLMs, achieving approximately 0.72-0.73 accuracy and comparable F1 scores. These findings highlight the challenges of training complex deep models on limited data and underscore the value of careful data augmentation and simpler algorithms (such as augmented SVM) for Bloom's Taxonomy classification.