Question-Score Identity Detection (Q-SID): A Statistical Algorithm to Detect Collusion Groups with Error Quantification from Exam Question Scores

TL;DR

Q-SID is a statistical algorithm that detects student collusion groups in various exam formats using only question scores, providing error quantification and demonstrating robust performance across diverse datasets.

Contribution

The paper introduces Q-SID, a novel statistical method for detecting collusion groups with error quantification, applicable to multiple exam formats and validated on real datasets.

Findings

Q-SID achieves low false-positive rates across diverse datasets.

It effectively detects collusion groups in various exam formats.

The method provides two types of error quantification for reliable detection.

Abstract

Collusion between students in online exams is a major problem that undermines the integrity of the exam results. Although there exist methods that use exam data to identify pairs of students who have likely copied each other's answers, these methods are restricted to specific formats of multiple-choice exams. Here we present a statistical algorithm, Q-SID, that efficiently detects groups of students who likely have colluded, i.e., collusion groups, with error quantification. Q-SID uses graded numeric question scores only, so it works for many formats of multiple-choice and non-multiple-choice exams. Q-SID reports two false-positive rates (FPRs) for each collusion group: (1) empirical FPR, whose null data are from 36 strictly proctored exam datasets independent of the user-input exam data and (2) synthetic FPR, whose null data are simulated from a copula-based probabilistic model, which is first fitted to the user-input exam data and then modified to have no collusion. On 34 unproctored exam datasets, including two benchmark datasets with true positives and negatives verified by textural analysis, we demonstrate that Q-SID is a collusion detection algorithm with powerful and robust performance across exam formats, numbers of questions and students, and exam complexity.