Reinforcement learning for pursuit and evasion of microswimmers at low Reynolds number

TL;DR

This paper models pursuit-evasion between microswimmers in a low-Reynolds-number fluid environment, demonstrating that adversarial reinforcement learning enables agents to develop complex strategies that outperform heuristics despite partial information.

Contribution

It introduces a reinforcement learning framework for pursuit-evasion in microswimmers, revealing how agents learn sophisticated tactics in a hydrodynamic setting with partial observability.

Findings

Agents outperform heuristic strategies.

Reinforcement learning discovers complex pursuit-evasion tactics.

Agents effectively handle partial information through learned behaviors.

Abstract

We consider a model of two competing microswimming agents engaged in a pursue-evasion task within a low-Reynolds-number environment. Agents can only perform simple maneuvers and sense hydrodynamic disturbances, which provide ambiguous (partial) information about the opponent's position and motion. We frame the problem as a zero-sum game: The pursuer has to capture the evader in the shortest time, while the evader aims at deferring capture as long as possible. We show that the agents, trained via adversarial reinforcement learning, are able to overcome partial observability by discovering increasingly complex sequences of moves and countermoves that outperform known heuristic strategies and exploit the hydrodynamic environment.