Learning to swim efficiently in a nonuniform flow field

TL;DR

This paper combines deep reinforcement learning with hydrodynamic simulations to study how microswimmers can learn to navigate efficiently in non-uniform flow fields using different sensory information.

Contribution

It introduces a novel approach integrating deep RL with hydrodynamics to analyze microswimmer navigation and sensory information requirements.

Findings

Lab frame orientation info suffices for certain navigation tasks.

Both translational and rotational velocity info are needed for flow direction navigation.

Local sensing of hydrodynamic forces enables comparable performance to lab frame sensing.

Abstract

Microswimmers can acquire information on the surrounding fluid by sensing mechanical queues. They can then navigate in response to these signals. We analyse this navigation by combining deep reinforcement learning with direct numerical simulations to resolve the hydrodynamics. We study how local and non-local information can be used to train a swimmer to achieve particular swimming tasks in a non-uniform flow field, in particular a zig-zag shear flow. The swimming tasks are (1) learning how to swim in the vorticity direction, (2) the shear-gradient direction, and (3) the shear flow direction. We find that access to lab frame information on the swimmer's instantaneous orientation is all that is required in order to reach the optimal policy for (1,2). However, information on both the translational and rotational velocities seem to be required to achieve (3). Inspired by biological microorganisms we also consider the case where the swimmers sense local information, i.e. surface hydrodynamic forces, together with a signal direction. This might correspond to gravity or, for micro-organisms with light sensors, a light source. In this case, we show that the swimmer can reach a comparable level of performance as a swimmer with access to lab frame variables. We also analyse the role of different swimming modes, i.e. pusher, puller, and neutral swimmers.