Stabilising viscous extensional flows using Reinforcement Learning

TL;DR

This paper develops a Reinforcement Learning-based control algorithm to stabilize a liquid drop in an unstable extensional flow created by a four-roll mill, demonstrating robustness and adaptability to various conditions.

Contribution

It introduces a probabilistic RL approach for stabilizing viscous extensional flows, outperforming previous control methods in robustness and adaptability.

Findings

RL successfully stabilizes the drop at the stagnation point.

The control algorithm is robust against thermal noise.

The method adapts to different initial positions and flow conditions.

Abstract

The four-roll mill, wherein four identical cylinders undergo rotation of identical magnitude but alternate signs, was originally proposed by GI Taylor to create local extensional flows and study their ability to deform small liquid drops. Since an extensional flow has an unstable eigendirection, a drop located at the flow stagnation point will have a tendency to escape. This unstable dynamics can however be stabilised using, e.g., a modulation of the rotation rates of the cylinders. Here we use Reinforcement Learning, a branch of Machine Learning devoted to the optimal selection of actions based on cumulative rewards, in order to devise a stabilisation algorithm for the four-roll mill flow. The flow is modelled as the linear superposition of four two-dimensional rotlets and the drop is treated as a rigid spherical particle smaller than all other length scales in the problem. Unlike previous attempts to devise control, we take a probabilistic approach whereby speed adjustments are drawn from a probability density function whose shape is improved over time via a form of gradient ascent know as Actor-Critic method. With enough training, our algorithm is able to precisely control the drop and keep it close to the stagnation point for as long as needed. We explore the impact of the physical and learning parameters on the effectiveness of the control and demonstrate the robustness of the algorithm against thermal noise. We finally show that Reinforcement Learning can provide a control algorithm effective for all initial positions and that can be adapted to limit the magnitude of the flow extension near the position of the drop.