Exceeding Information Targets in Fixed-Form Test Assembly

TL;DR

This paper introduces a novel test assembly method that constructs tests with information functions exceeding a target, ensuring lower ability estimation error, and demonstrates that such tests are far more numerous than those meeting the target.

Contribution

It proposes a new approach to fixed-form test assembly by generating tests with information functions that surpass the target, enhancing test quality and diversity.

Findings

Number of exceeding tests is much greater than meeting tests.

Monte Carlo importance sampling effectively assembles target exceeding tests.

Exceeding tests guarantee lower ability estimation error.

Abstract

This work studies the assembly of ability estimation test forms (called tests, for short) drawn from an item bank. The goal of fixed-from test assembly is to generate a large number of different tests with information functions that meet a target information function. Thus, every test has the same ability estimation error. This work proposes a new way of test assembly, namely, drawing tests with information functions that exceed the target. This guarantees that every test has an ability estimation error that is less than the error set by the target. The work estimates the number of target exceeding tests as a function of the number of items in the test. It demonstrates that the number of target exceeding tests is far greater than the number of target meeting tests. A Monte Carlo importance sampling algorithm is proposed for target exceeding test assembly.