Collaborative and AI-aided Exam Question Generation using Wikidata in Education

TL;DR

This paper introduces PhysWikiQuiz, a multilingual, collaborative AI-powered system that generates, tests, and corrects physics questions using Wikidata, significantly aiding teachers in creating personalized exam questions efficiently.

Contribution

It presents a novel Wikimedia-based framework integrating collaborative knowledge engineering with AI for automatic physics question generation and validation.

Findings

System can generate and correct up to 300 questions per formula.

Effective in verifying answer values and units using CAS.

Works with a single formula concept to produce diverse questions.

Abstract

Since the COVID-19 outbreak, the use of digital learning or education platforms has significantly increased. Teachers now digitally distribute homework and provide exercise questions. In both cases, teachers need to continuously develop novel and individual questions. This process can be very time-consuming and should be facilitated and accelerated both through exchange with other teachers and by using Artificial Intelligence (AI) capabilities. To address this need, we propose a multilingual Wikimedia framework that allows for collaborative worldwide teacher knowledge engineering and subsequent AI-aided question generation, test, and correction. As a proof of concept, we present >>PhysWikiQuiz<<, a physics question generation and test engine. Our system (hosted by Wikimedia at https://physwikiquiz.wmflabs.org) retrieves physics knowledge from the open community-curated database Wikidata. It can generate questions in different variations and verify answer values and units using a Computer Algebra System (CAS). We evaluate the performance on a public benchmark dataset at each stage of the system workflow. For an average formula with three variables, the system can generate and correct up to 300 questions for individual students based on a single formula concept name as input by the teacher.