Navigation of a Three-Link Microswimmer via Deep Reinforcement Learning

TL;DR

This paper demonstrates how reinforcement learning can be used to develop adaptive and efficient stroke patterns for a microswimmer, enabling complex navigation tasks at low Reynolds numbers.

Contribution

It introduces RL-based strategies for microswimmer control, comparing them with traditional methods and showing their adaptability in complex navigation scenarios.

Findings

RL can generate effective stroke patterns for microswimmers.

Different reward functions influence stroke development.

RL enables microswimmers to perform complex navigation tasks.

Abstract

Motile microorganisms develop effective swimming gaits to adapt to complex biological environments. Translating this adaptability to smart microrobots presents significant challenges in motion planning and stroke design. In this work, we explore the use of reinforcement learning (RL) to develop stroke patterns for targeted navigation in a three-link swimmer model at low Reynolds numbers. Specifically, we design two RL-based strategies: one focusing on maximizing velocity (Velocity-Focused Strategy) and another balancing velocity with energy consumption (Energy-Aware Strategy). Our results demonstrate how the use of different reward functions influences the resulting stroke patterns developed via RL, which are compared with those obtained from traditional optimization methods. Furthermore, we showcase the capability of the RL-powered swimmer in adapting its stroke patterns in performing different navigation tasks, including tracing complex trajectories and pursuing moving targets. Taken together, this work highlights the potential of reinforcement learning as a versatile tool for designing efficient and adaptive microswimmers capable of sophisticated maneuvers in complex environments.