Hierarchical Planning for Dynamic Resource Allocation in Smart and Connected Communities

TL;DR

This paper introduces a hierarchical planning approach for dynamic resource allocation in city-scale cyber-physical systems, improving scalability and effectiveness in emergency response scenarios.

Contribution

It presents a novel hierarchical framework that decomposes large resource allocation problems into smaller ones solved via Monte-Carlo planning, validated with real city data.

Findings

Outperforms existing emergency response methods

Effective in large-scale city environments

Scalable and adaptable to real-world data

Abstract

Resource allocation under uncertainty is a classical problem in city-scale cyber-physical systems. Consider emergency response as an example; urban planners and first responders optimize the location of ambulances to minimize expected response times to incidents such as road accidents. Typically, such problems deal with sequential decision-making under uncertainty and can be modeled as Markov (or semi-Markov) decision processes. The goal of the decision-maker is to learn a mapping from states to actions that can maximize expected rewards. While online, offline, and decentralized approaches have been proposed to tackle such problems, scalability remains a challenge for real-world use-cases. We present a general approach to hierarchical planning that leverages structure in city-level CPS problems for resource allocation. We use emergency response as a case study and show how a large resource allocation problem can be split into smaller problems. We then use Monte-Carlo planning for solving the smaller problems and managing the interaction between them. Finally, we use data from Nashville, Tennessee, a major metropolitan area in the United States, to validate our approach. Our experiments show that the proposed approach outperforms state-of-the-art approaches used in the field of emergency response.