MOLTE: a Modular Optimal Learning Testing Environment

TL;DR

MOLTE is a modular, Matlab-based testing environment designed to facilitate comprehensive empirical evaluation of Bayesian learning algorithms across diverse problems, supporting easy comparison, customization, and scalability.

Contribution

It introduces a flexible, open-source platform for testing and comparing various learning policies on a wide range of problems, enhancing research capabilities in optimal learning.

Findings

Demonstrated MOLTE's ability to compare policies effectively.

Showcased the platform's scalability with parallel computing.

Addressed prior gaps in tuning and prior construction in the literature.

Abstract

We address the relative paucity of empirical testing of learning algorithms (of any type) by introducing a new public-domain, Modular, Optimal Learning Testing Environment (MOLTE) for Bayesian ranking and selection problem, stochastic bandits or sequential experimental design problems. The Matlab-based simulator allows the comparison of a number of learning policies (represented as a series of .m modules) in the context of a wide range of problems (each represented in its own .m module) which makes it easy to add new algorithms and new test problems. State-of-the-art policies and various problem classes are provided in the package. The choice of problems and policies is guided through a spreadsheet-based interface. Different graphical metrics are included. MOLTE is designed to be compatible with parallel computing to scale up from local desktop to clusters and clouds. We offer MOLTE as an easy-to-use tool for the research community that will make it possible to perform much more comprehensive testing, spanning a broader selection of algorithms and test problems. We demonstrate the capabilities of MOLTE through a series of comparisons of policies on a starter library of test problems. We also address the problem of tuning and constructing priors that have been largely overlooked in optimal learning literature. We envision MOLTE as a modest spur to provide researchers an easy environment to study interesting questions involved in optimal learning.