Optimizing Heat Alert Issuance with Reinforcement Learning

TL;DR

This paper develops and evaluates a reinforcement learning framework to optimize heat alert systems, aiming to reduce heat-related health impacts by using a new environment and data-driven policies.

Contribution

It introduces a novel RL environment for heat alert policy evaluation, incorporating comprehensive data and tailored Bayesian models, and analyzes RL performance and policy improvements.

Findings

RL policies can outperform current alert systems in certain scenarios

Policy constraints improve RL effectiveness in heat alert issuance

Contrastive analysis reveals conditions for significant policy gains

Abstract

A key strategy in societal adaptation to climate change is using alert systems to prompt preventative action and reduce the adverse health impacts of extreme heat events. This paper implements and evaluates reinforcement learning (RL) as a tool to optimize the effectiveness of such systems. Our contributions are threefold. First, we introduce a new publicly available RL environment enabling the evaluation of the effectiveness of heat alert policies to reduce heat-related hospitalizations. The rewards model is trained from a comprehensive dataset of historical weather, Medicare health records, and socioeconomic/geographic features. We use scalable Bayesian techniques tailored to the low-signal effects and spatial heterogeneity present in the data. The transition model uses real historical weather patterns enriched by a data augmentation mechanism based on climate region similarity. Second, we use this environment to evaluate standard RL algorithms in the context of heat alert issuance. Our analysis shows that policy constraints are needed to improve RL's initially poor performance. Third, a post-hoc contrastive analysis provides insight into scenarios where our modified heat alert-RL policies yield significant gains/losses over the current National Weather Service alert policy in the United States.