Stochastic Dynamic Network Utility Maximization with Application to Disaster Response

TL;DR

This paper develops a stochastic dynamic network utility maximization framework for disaster response, integrating hierarchical decision-making, deep reinforcement learning, and primal-dual algorithms to optimize resource allocation under uncertainty.

Contribution

It introduces a novel decomposition approach combining primal-dual methods with deep reinforcement learning for complex, stochastic disaster response scenarios.

Findings

Effective resource allocation in complex disaster environments

Decoupling local utilities from central decision-making

Utilizing deep RL for modeling local responses

Abstract

In this paper, we are interested in solving Network Utility Maximization (NUM) problems whose underlying local utilities and constraints depend on a complex stochastic dynamic environment. While the general model applies broadly, this work is motivated by resource sharing during disasters concurrently occurring in multiple areas. In such situations, hierarchical layers of Incident Command Systems (ICS) are engaged; specifically, a central entity (e.g., the federal government) typically coordinates the incident response allocating resources to different sites, which then get distributed to the affected by local entities. The benefits of an allocation decision to the different sites are generally not expressed explicitly as a closed-form utility function because of the complexity of the response and the random nature of the underlying phenomenon we try to contain. We use the classic approach of decomposing the NUM formulation and applying a primal-dual algorithm to achieve optimal higher-level decisions under coupled constraints while modeling the optimized response to the local dynamics with deep reinforcement learning algorithms. The decomposition we propose has several benefits: 1) the entities respond to their local utilities based on a congestion signal conveyed by the ICS upper layers; 2) the complexity of capturing the utility of local responses and their diversity is addressed effectively without sharing local parameters and priorities with the ICS layers above; 3) utilities, known as explicit functions, are approximated as convex functions of the resources allocated; 4) decisions rely on up-to-date data from the ground along with future forecasts.