An Analysis of Reinforcement Learning for Malaria Control

TL;DR

This paper systematically analyzes various reinforcement learning formulations for malaria control, demonstrating that simple algorithms like Upper Confidence Bounds are effective and often outperform more complex methods in this domain.

Contribution

It provides a comprehensive comparison of different RL formulations and algorithms for malaria control, highlighting the effectiveness of simple UCB-based methods.

Findings

UCB algorithms perform well in malaria policy learning

Simple algorithms can outperform complex RL methods

Analysis covers multiple formulations and compares to black box optimization

Abstract

Previous work on policy learning for Malaria control has often formulated the problem as an optimization problem assuming the objective function and the search space have a specific structure. The problem has been formulated as multi-armed bandits, contextual bandits and a Markov Decision Process in isolation. Furthermore, an emphasis is put on developing new algorithms specific to an instance of Malaria control, while ignoring a plethora of simpler and general algorithms in the literature. In this work, we formally study the formulation of Malaria control and present a comprehensive analysis of several formulations used in the literature. In addition, we implement and analyze several reinforcement learning algorithms in all formulations and compare them to black box optimization. In contrast to previous work, our results show that simple algorithms based on Upper Confidence Bounds are sufficient for learning good Malaria policies, and tend to outperform their more advanced counterparts on the malaria OpenAI Gym environment.