Double Q-Learning for Citizen Relocation During Natural Hazards

TL;DR

This paper explores using Double Q-learning within a POMDP framework to enable autonomous robots to relocate citizens during natural hazards, demonstrating variable success rates depending on scenario difficulty.

Contribution

It introduces a novel application of Double Q-learning for citizen relocation in disaster scenarios using a grid world simulation.

Findings

Success rate above 100% in easy scenarios

Near 50% success rate in hard scenarios

Demonstrates potential of RL in autonomous disaster response

Abstract

Natural disasters can cause substantial negative socio-economic impacts around the world, due to mortality, relocation, rates, and reconstruction decisions. Robotics has been successfully applied to identify and rescue victims during the occurrence of a natural hazard. However, little effort has been taken to deploy solutions where an autonomous robot can save the life of a citizen by itself relocating it, without the need to wait for a rescue team composed of humans. Reinforcement learning approaches can be used to deploy such a solution, however, one of the most famous algorithms to deploy it, the Q-learning, suffers from biased results generated when performing its learning routines. In this research a solution for citizen relocation based on Partially Observable Markov Decision Processes is adopted, where the capability of the Double Q-learning in relocating citizens during a natural hazard is evaluated under a proposed hazard simulation engine based on a grid world. The performance of the solution was measured as a success rate of a citizen relocation procedure, where the results show that the technique portrays a performance above 100% for easy scenarios and near 50% for hard ones.