Attention-Driven Hierarchical Reinforcement Learning with Particle Filtering for Source Localization in Dynamic Fields

TL;DR

This paper introduces a hierarchical reinforcement learning framework with attention-driven particle filtering to improve source localization in dynamic, noisy environments, demonstrating enhanced accuracy and efficiency.

Contribution

It presents a novel integration of attention-enhanced particle filtering with hierarchical reinforcement learning for dynamic source localization, with proven convergence and superior experimental performance.

Findings

Proven convergence of the attention particle filter.

Superior accuracy and adaptability in diverse scenarios.

Enhanced computational efficiency over traditional methods.

Abstract

In many real-world scenarios, such as gas leak detection or environmental pollutant tracking, solving the Inverse Source Localization and Characterization problem involves navigating complex, dynamic fields with sparse and noisy observations. Traditional methods face significant challenges, including partial observability, temporal and spatial dynamics, out-of-distribution generalization, and reward sparsity. To address these issues, we propose a hierarchical framework that integrates Bayesian inference and reinforcement learning. The framework leverages an attention-enhanced particle filtering mechanism for efficient and accurate belief updates, and incorporates two complementary execution strategies: Attention Particle Filtering Planning and Attention Particle Filtering Reinforcement Learning. These approaches optimize exploration and adaptation under uncertainty. Theoretical analysis proves the convergence of the attention-enhanced particle filter, while extensive experiments across diverse scenarios validate the framework's superior accuracy, adaptability, and computational efficiency. Our results highlight the framework's potential for broad applications in dynamic field estimation tasks.