The Achievement of Higher Flexibility in Multiple Choice-based Tests Using Image Classification Techniques

TL;DR

This paper introduces a machine learning-based system for recognizing and classifying answer boxes in MCQ tests, reducing restrictions of traditional optical mark reading systems and improving accuracy across diverse answer patterns.

Contribution

It presents a novel approach using image registration and machine learning classifiers, including CNNs, to distinguish confirmed, crossed out, and blank answers with minimal constraints.

Findings

High accuracy in answer classification across diverse answer patterns

Effective use of a new dataset with six real MCQ assessments

Reduced restrictions compared to traditional OMR systems

Abstract

In spite of the high accuracy of the existing optical mark reading (OMR) systems and devices, a few restrictions remain existent. In this work, we aim to reduce the restrictions of multiple choice questions (MCQ) within tests. We use an image registration technique to extract the answer boxes from answer sheets. Unlike other systems that rely on simple image processing steps to recognize the extracted answer boxes, we address the problem from another perspective by training a machine learning classifier to recognize the class of each answer box (i.e., confirmed, crossed out, or blank answer). This gives us the ability to deal with a variety of shading and mark patterns, and distinguish between chosen (i.e., confirmed) and canceled answers (i.e., crossed out). All existing machine learning techniques require a large number of examples in order to train a model for classification, therefore we present a dataset including six real MCQ assessments with different answer sheet templates. We evaluate two strategies of classification: a straight-forward approach and a two-stage classifier approach. We test two handcrafted feature methods and a convolutional neural network. In the end, we present an easy-to-use graphical user interface of the proposed system. Compared with existing OMR systems, the proposed system has the least constraints and achieves a high accuracy. We believe that the presented work will further direct the development of OMR systems towards reducing the restrictions of the MCQ tests.