Detection of Two-Way Outliers in Multivariate Data and Application to Cheating Detection in Educational Tests

TL;DR

This paper introduces a new Bayesian latent variable model for detecting outliers among individuals and items in multivariate data, specifically applied to identify cheating in educational tests, with demonstrated effectiveness through simulations and a case study.

Contribution

A novel latent variable model combining factor analysis and latent class modeling for two-way outlier detection in item-response data.

Findings

Effective detection of cheating in educational tests.

Model outperforms existing methods in simulations.

Provides a Bayesian decision framework with controlled error rates.

Abstract

The paper proposes a new latent variable model for the simultaneous (two-way) detection of outlying individuals and items for item-response-type data. The proposed model is a synergy between a factor model for binary responses and continuous response times that captures normal item response behaviour and a latent class model that captures the outlying individuals and items. A statistical decision framework is developed under the proposed model that provides compound decision rules for controlling local false discovery/nondiscovery rates of outlier detection. Statistical inference is carried out under a Bayesian framework, for which a Markov chain Monte Carlo algorithm is developed. The proposed method is applied to the detection of cheating in educational tests due to item leakage using a case study of a computer-based nonadaptive licensure assessment. The performance of the proposed method is evaluated by simulation studies.