Optimal navigation of magnetic artificial microswimmers in blood capillaries with deep reinforcement learning

TL;DR

This paper develops a deep reinforcement learning-based control policy for navigating magnetic artificial microswimmers through complex blood capillaries, enabling precise targeted delivery with reduced computational effort.

Contribution

It introduces a novel reinforcement learning approach coupled with a reduced-order blood flow model to robustly control microswimmers in realistic capillary networks.

Findings

Reinforcement learning policy successfully guides microswimmers to targets.

Policy is robust across different blood flow simulations.

Approach reduces computational costs for navigation planning.

Abstract

Biomedical applications such as targeted drug delivery, microsurgery, and sensing rely on reaching precise areas within the body in a minimally invasive way. Artificial bacterial flagella (ABFs) have emerged as potential tools for this task by navigating through the circulatory system with the help of external magnetic fields. While their swimming characteristics are well understood in simple settings, their controlled navigation through realistic capillary networks remains a significant challenge due to the complexity of blood flow and the high computational cost of detailed simulations. We address this challenge by conducting numerical simulations of ABFs in retinal capillaries, propelled by an external magnetic field. The simulations are based on a validated blood model that predicts the dynamics of individual red blood cells and their hydrodynamic interactions with ABFs. The magnetic field follows a control policy that brings the ABF to a prescribed target. The control policy is learned with an actor-critic, off-policy reinforcement learning algorithm coupled with a reduced-order model of the system. We show that the same policy robustly guides the ABF to a prescribed target in both the reduced-order model and the fine-grained blood simulations. This approach is suitable for designing robust control policies for personalized medicine at moderate computational cost.