# ML-based validation of experimental randomization in learning games

## TL;DR

This paper explores using machine learning to check if randomization in experiments is valid, finding that some models work better than others.

## Contribution

The novelty lies in applying ML models to validate experimental randomization and identifying factors affecting their effectiveness.

## Key findings

- Supervised ML models achieved 87% accuracy in detecting randomization patterns after synthetic data augmentation.
- Unsupervised models like k-means and ANN performed poorly, with ANN showing overfitting issues.
- Feature importance analysis identified predictors of assignment bias in experimental designs.

## Abstract

Randomization is a standard method in experimental research, yet its validity is not always guaranteed. This study introduces machine learning (ML) models as supplementary tools for validating participant randomization. A learning direction game with dichotomized scenarios was introduced, and both supervised and unsupervised ML models were evaluated on a binary classification task. Supervised models (logistic regression, decision tree, and support vector machine) achieved the highest accuracy of 87% after adding synthetic data to enlarge the sample size, while unsupervised models (k-means, k-nearest neighbors, and ANN—artificial neural networks) performed less effectively. The ANN model, in particular, showed overfitting, even with synthetic data. Feature importance analysis further revealed predictors of assignment bias. These findings support the proposed methodology for detecting randomization patterns; however, its effectiveness is influenced by sample size and experimental design complexity. Future studies should apply this approach with caution and further examine its applicability across diverse experimental designs.