Programmable Control of Ultrasound Swarmbots through Reinforcement Learning

TL;DR

This paper demonstrates autonomous acoustic navigation of microbubbles using reinforcement learning, enabling intelligent control of micro/nanorobots for targeted drug delivery in microfluidic environments.

Contribution

It introduces reinforcement learning for controlling acoustically powered microbubbles, achieving autonomous navigation and swarm manipulation for the first time.

Findings

Successful autonomous microbubble navigation in microfluidic environments

Controlled microbubbles swarm formation and trajectory following

Robust control validated with extensive image-based training

Abstract

Powered by acoustics, existing therapeutic and diagnostic procedures will become less invasive and new methods will become available that have never been available before. Acoustically driven microrobot navigation based on microbubbles is a promising approach for targeted drug delivery. Previous studies have used acoustic techniques to manipulate microbubbles in vitro and in vivo for the delivery of drugs using minimally invasive procedures. Even though many advanced capabilities and sophisticated control have been achieved for acoustically powered microrobots, there remain many challenges that remain to be solved. In order to develop the next generation of intelligent micro/nanorobots, it is highly desirable to conduct accurate identification of the micro-nanorobots and to control their dynamic motion autonomously. Here we use reinforcement learning control strategies to learn the microrobot dynamics and manipulate them through acoustic forces. The result demonstrated for the first time autonomous acoustic navigation of microbubbles in a microfluidic environment. Taking advantage of the benefit of the second radiation force, microbubbles swarm to form a large swarm, which is then driven along the desired trajectory. More than 100 thousand images were used for the training to study the unexpected dynamics of microbubbles. As a result of this work, the microrobots are validated to be controlled, illustrating a good level of robustness and providing computational intelligence to the microrobots, which enables them to navigate independently in an unstructured environment without requiring outside assistance.