Multi-step dual control for exploration and exploitation in autonomous search with convergence guarantee

TL;DR

This paper introduces a Multi-Step Dual Control framework for autonomous source search that guarantees convergence, effectively balancing exploration and exploitation to improve search success and efficiency in uncertain environments.

Contribution

The paper proposes a novel Multi-Step DCEE approach with convergence guarantees, addressing challenges of unknown source and environment in autonomous search.

Findings

Outperforms SMPC, IPP, and single-step DCEE in success rates

Demonstrates improved search efficiency in simulations

Guarantees recursive feasibility and convergence

Abstract

Motivated by the recently proposed dual control for exploration and exploitation (DCEE) concept, this paper presents a Multi-Step DCEE (MS-DCEE) framework with guaranteed convergence for autonomous search of a source of airborne dispersion. Different from the existing stochastic model predictive control (SMPC) algorithm and informative path planning (IPP) approaches, the proposed MS-DCEE approach uses the current and future input to not only drive the agent towards the estimated source location (exploitation) but also reduce its estimation uncertainty (exploration) by actively learning the operational environment. Unknown source target position, together with unknown environment, impose significant challenges in establishing the recursive feasibility and the convergence of the proposed algorithm. To address them, with the help of the property of Bayesian estimation, we develop a two-step approach where the unbiasedness of the mean estimation is assumed first and then the randomness of the mean estimate under each collected information sequence is accounted. Based on that, we develop a MS-DCEE scheme with suitable terminal ingredients where recursive feasibility and convergence are guaranteed. Two simulation scenarios are conducted, which show that the proposed MS-DCEE algorithm outperforms the SMPC, the IPP and the single-step DCEE approaches in terms of searching successful rates and efficiency.