Emergence of a Flow-Assisted Casting Strategy for Olfactory Navigation via Memory-Augmented Reinforcement Learning

TL;DR

This paper investigates how reinforcement learning agents develop a flow-assisted casting strategy for olfactory navigation in unsteady flows, highlighting the role of memory length in optimizing search success.

Contribution

It demonstrates that RL agents can autonomously learn a flow-assisted casting strategy and adapt their search behavior without predefined models, influenced by memory length and flow conditions.

Findings

Agents develop a flow-assisted casting strategy.

Search success depends non-monotonically on memory length.

The sector-search model explains the speed dependence.

Abstract

In dynamic flow fields, various animals exhibit remarkable odor search capabilities despite relying on stochastic detections. Interestingly, there exists an optimal time window for integrating these detections that maximizes search efficiency. To understand the underlying mechanism, we investigate the navigation performance of Reinforcement Learning (RL) agents in unsteady flows under varying memory lengths and flow conditions. Without any predefined models, the agents develop a flow-assisted casting strategy and adaptively adjust both the geometry of their search trajectories and the concentration threshold for initiating casting to maximize the success rate. The agent's average speed toward the odor source exhibits a non-monotonic dependence on memory length, which can be explained by the "sector-search" model.