Multi-source Plume Tracing via Multi-Agent Reinforcement Learning

TL;DR

This paper introduces a multi-agent reinforcement learning approach for localizing multiple airborne pollution sources using UAV swarms, outperforming traditional methods in complex, turbulent environments.

Contribution

The paper presents a novel MARL algorithm modeled as a POMG with an LSTM-based ADDRQN, incorporating realistic environmental factors and action histories for improved plume source localization.

Findings

Agents explore only 1.29% of the environment to locate sources.

The method outperforms conventional approaches in simulated environments.

Incorporates realistic turbulence and sensor noise in the simulation.

Abstract

Industrial catastrophes like the Bhopal disaster (1984) and the Aliso Canyon gas leak (2015) demonstrate the urgent need for rapid and reliable plume tracing algorithms to protect public health and the environment. Traditional methods, such as gradient-based or biologically inspired approaches, often fail in realistic, turbulent conditions. To address these challenges, we present a Multi-Agent Reinforcement Learning (MARL) algorithm designed for localizing multiple airborne pollution sources using a swarm of small uncrewed aerial systems (sUAS). Our method models the problem as a Partially Observable Markov Game (POMG), employing a Long Short-Term Memory (LSTM)-based Action-specific Double Deep Recurrent Q-Network (ADDRQN) that uses full sequences of historical action-observation pairs, effectively approximating latent states. Unlike prior work, we use a general-purpose simulation environment based on the Gaussian Plume Model (GPM), incorporating realistic elements such as a three-dimensional environment, sensor noise, multiple interacting agents, and multiple plume sources. The incorporation of action histories as part of the inputs further enhances the adaptability of our model in complex, partially observable environments. Extensive simulations show that our algorithm significantly outperforms conventional approaches. Specifically, our model allows agents to explore only 1.29\% of the environment to successfully locate pollution sources.