How to Control Hydrodynamic Force on Fluidic Pinball via Deep Reinforcement Learning

TL;DR

This paper demonstrates a deep reinforcement learning approach for real-time control of hydrodynamic forces on a fluidic pinball system, outperforming brute-force methods and providing insights into flow control mechanisms.

Contribution

It introduces a DRL-based control strategy for fluidic pinball that effectively manages hydrodynamic forces and offers a novel analysis of decision-making and physical mechanisms.

Findings

DRL achieves better control than brute-force search.

The control strategy is effective in nonparametric spaces.

Insights into physical mechanisms of force tracking.

Abstract

Deep reinforcement learning (DRL) for fluidic pinball, three individually rotating cylinders in the uniform flow arranged in an equilaterally triangular configuration, can learn the efficient flow control strategies due to the validity of self-learning and data-driven state estimation for complex fluid dynamic problems. In this work, we present a DRL-based real-time feedback strategy to control the hydrodynamic force on fluidic pinball, i.e., force extremum and tracking, from cylinders' rotation. By adequately designing reward functions and encoding historical observations, and after automatic learning of thousands of iterations, the DRL-based control was shown to make reasonable and valid control decisions in nonparametric control parameter space, which is comparable to and even better than the optimal policy found through lengthy brute-force searching. Subsequently, one of these results was analyzed by a machine learning model that enabled us to shed light on the basis of decision-making and physical mechanisms of the force tracking process. The finding from this work can control hydrodynamic force on the operation of fluidic pinball system and potentially pave the way for exploring efficient active flow control strategies in other complex fluid dynamic problems.