Bayesian decision theory for tree-based adaptive screening tests with an application to youth delinquency

TL;DR

This paper introduces a Bayesian decision theory framework for designing efficient tree-based adaptive screening tests, demonstrated through youth delinquency risk assessment, balancing test brevity and accuracy.

Contribution

It develops a novel Bayesian decision framework and a new method for designing adaptive tests using classification trees, addressing the lack of principled test termination criteria.

Findings

Adaptive tests with fewer than 10 questions nearly match the accuracy of 173-item surveys.

The framework effectively balances test length and accuracy in risk assessment.

Application to youth delinquency shows practical utility of the method.

Abstract

Crime prevention strategies based on early intervention depend on accurate risk assessment instruments for identifying high risk youth. It is important in this context that the instruments be convenient to administer, which means, in particular, that they should also be reasonably brief; adaptive screening tests are useful for this purpose. Adaptive tests constructed using classification and regression trees are becoming a popular alternative to traditional Item Response Theory (IRT) approaches for adaptive testing. However, tree-based adaptive tests lack a principled criterion for terminating the test. This paper develops a Bayesian decision theory framework for measuring the trade-off between brevity and accuracy, when considering tree-based adaptive screening tests of different lengths. We also present a novel method for designing tree-based adaptive tests, motivated by this framework. The framework and associated adaptive test method are demonstrated through an application to youth delinquency risk assessment in Honduras; it is shown that an adaptive test requiring a subject to answer fewer than 10 questions can identify high risk youth nearly as accurately as an unabridged survey containing 173 items.