A New Score for Adaptive Tests in Bayesian and Credal Networks

TL;DR

This paper introduces a new, more explainable scoring method for adaptive tests using credal networks, which handle uncertainty more effectively and simplify the evaluation process without compromising performance.

Contribution

It proposes a mode-based scoring approach for credal networks in adaptive testing, improving explainability and computational simplicity over traditional information-theoretic scores.

Findings

Mode-based scores are easier to interpret and compute.

The new scores perform comparably to traditional methods in tests.

Numerical experiments validate the effectiveness of the approach.

Abstract

A test is adaptive when its sequence and number of questions is dynamically tuned on the basis of the estimated skills of the taker. Graphical models, such as Bayesian networks, are used for adaptive tests as they allow to model the uncertainty about the questions and the skills in an explainable fashion, especially when coping with multiple skills. A better elicitation of the uncertainty in the question/skills relations can be achieved by interval probabilities. This turns the model into a credal network, thus making more challenging the inferential complexity of the queries required to select questions. This is especially the case for the information theoretic quantities used as scores to drive the adaptive mechanism. We present an alternative family of scores, based on the mode of the posterior probabilities, and hence easier to explain. This makes considerably simpler the evaluation in the credal case, without significantly affecting the quality of the adaptive process. Numerical tests on synthetic and real-world data are used to support this claim.