On the Advances and Challenges of Adaptive Online Testing

TL;DR

This paper provides a comprehensive overview of adaptive online testing, discussing its components, recent developments, challenges, and future research directions in a systematic manner.

Contribution

It offers a systematic and complete treatment of adaptive online testing, including exploration, inference, and analysis, filling a gap in the current literature.

Findings

Overview of adaptive online testing components

Discussion of recent developments and research directions

Identification of challenges and future research areas

Abstract

In recent years, the interest in developing adaptive solutions for online testing has grown significantly in the industry. While the advances related to this relative new technology have been developed in multiple domains, it lacks in the literature a systematic and complete treatment of the procedure that involves exploration, inference, and analysis. This short paper aims to develop a comprehensive understanding of adaptive online testing, including various building blocks and analytical results. We also address the latest developments, research directions, and challenges that have been less mentioned in the literature.