# Calculating Bias in Test Score Equating in a NEAT Design

## TL;DR

This paper compares how different methods of equating test scores affect bias in college admissions tests.

## Contribution

The study shows that the choice of criterion function significantly impacts equating bias conclusions.

## Key findings

- Bias definitions heavily influence which equating method is preferred in different scenarios.
- Practical implications for standardized tests are provided based on the simulation and real data analysis.
- Recommendations are given for calculating bias when evaluating equating transformations.

## Abstract

Test score equating is used to make scores from different test forms comparable, even when groups differ in ability. In practice, the non-equivalent group with anchor test (NEAT) design is commonly used. The overall aim was to compare the amount of bias under different conditions when using either chained equating or frequency estimation with five different criterion functions: the identity function, linear equating, equipercentile, chained equating and frequency estimation. We used real test data from a multiple-choice binary scored college admissions test to illustrate that the choice of criterion function matter. Further, we simulated data in line with the empirical data to examine difference in ability between groups, difference in item difficulty, difference in anchor test form and regular test form length, difference in correlations between anchor test form and regular test forms, and different sample size. The results indicate that how bias is defined heavily affects the conclusions we draw about which equating method is to be preferred in different scenarios. Practical implications of this in standardized tests are given together with recommendations on how to calculate bias when evaluating equating transformations.