Clock-state olfactory search in turbulent flows using Q-learning: The geometry of plume recovery

TL;DR

This paper presents a Q-learning based olfactory search strategy in turbulent flows, inspired by insect behaviors, demonstrating effective plume recovery with minimal memory but limited adaptability to turbulence variability.

Contribution

It introduces a simple, interpretable Q-learning approach for odor source localization that mimics insect behaviors and discusses its limitations and potential improvements.

Findings

The agent successfully recovers plumes using minimal memory.

Performance is good on turbulence simulation data.

Adding flexibility enhances robustness to intermittency.

Abstract

Finding an odor source in a turbulent flow requires effectively leveraging the history of olfactory observations into a robust navigation strategy. In this work, we use tabular Q-learning to train an olfactory search agent with a minimal memory of past observations: only a running clock since the last whiff. This agent learns an interpretable strategy to recover the plume which combines well-known behaviors observed in insects: surging, casting, and a return downwind. While achieving good performance on data from direct numerical simulations of turbulence, the agent is limited by an inability to adapt its strategy to the local intermittency level; we show that providing more flexibility improves robustness.