Smart Magnetic Microrobots Learn to Swim with Deep Reinforcement Learning

TL;DR

This paper demonstrates how deep reinforcement learning enables magnetic microrobots to autonomously learn effective swimming control policies in complex, unstructured environments, significantly advancing microrobot autonomy and adaptability.

Contribution

It introduces a novel application of deep reinforcement learning to control magnetic microrobots, achieving efficient autonomous learning and policy fine-tuning in biomimetic fluidic environments.

Findings

Reinforcement learning enabled successful microrobot control with fewer than 100,000 training steps.

The learned policies can be fine-tuned via regression to fit mathematical functions.

Deep RL significantly enhances microrobot adaptability in complex environments.

Abstract

Swimming microrobots are increasingly developed with complex materials and dynamic shapes and are expected to operate in complex environments in which the system dynamics are difficult to model and positional control of the microrobot is not straightforward to achieve. Deep reinforcement learning is a promising method of autonomously developing robust controllers for creating smart microrobots, which can adapt their behavior to operate in uncharacterized environments without the need to model the system dynamics. Here, we report the development of a smart helical magnetic hydrogel microrobot that used the soft actor critic reinforcement learning algorithm to autonomously derive a control policy which allowed the microrobot to swim through an uncharacterized biomimetic fluidic environment under control of a time varying magnetic field generated from a three-axis array of electromagnets. The reinforcement learning agent learned successful control policies with fewer than 100,000 training steps, demonstrating sample efficiency for fast learning. We also demonstrate that we can fine tune the control policies learned by the reinforcement learning agent by fitting mathematical functions to the learned policy's action distribution via regression. Deep reinforcement learning applied to microrobot control is likely to significantly expand the capabilities of the next generation of microrobots.