Position control of an acoustic cavitation bubble by reinforcement learning

TL;DR

This paper presents a reinforcement learning-based control method for precisely manipulating the position of an acoustic cavitation bubble within a standing wave field, achieving faster control than traditional linear theory methods.

Contribution

It introduces a novel reinforcement learning approach using Deep Deterministic Policy Gradient for continuous pressure control of cavitation bubbles, enabling faster and more precise positioning.

Findings

Control speed can be up to 7 times faster than linear theory predictions.

The RL agent effectively minimizes bubble-target distance with continuous pressure adjustments.

The method supports arbitrary control within a specified spatial range.

Abstract

A control technique is developed via Reinforcement Learning that allows arbitrary controlling of the position of an acoustic cavitation bubble in a dual-frequency standing acoustic wave field. The agent must choose the optimal pressure amplitude values to manipulate the bubble position in the range of $x/\lambda_0\in[0.05, 0.25]$. To train the agent an actor-critic off-policy algorithm (Deep Deterministic Policy Gradient) was used that supports continuous action space, which allows setting the pressure amplitude values continuously within $0$ and $1\, \mathrm{bar}$. A shaped reward function is formulated that minimizes the distance between the bubble and the target position and implicitly encourages the agent to perform the position control within the shortest amount of time. In some cases, the optimal control can be 7 times faster than the solution expected from the linear theory.