Robust flow control and optimal sensor placement using deep reinforcement learning

TL;DR

This paper demonstrates a deep reinforcement learning approach for drag reduction in laminar flow around a cylinder, achieving energy-efficient, robust control and optimized sensor placement, with potential for practical flow control applications.

Contribution

It introduces a novel reinforcement learning algorithm (S-PPO-CMA) for sensor placement optimization and demonstrates robust, energy-efficient flow control in a simulated environment.

Findings

Drag reduced by 18.4% at Re=120

Control strategy robust to Re in [100,216] and noise levels as low as 0.2

Optimized sensor layout with only 5 sensors maintaining high performance

Abstract

This paper focuses on a drag-reducing control strategy on a 2D-simulated laminar flow past a cylinder. Deep reinforcement learning algorithms have been implemented to discover efficient control schemes, using two synthetic jets located on the cylinder's poles as actuators and pressure sensors in the wake of the cylinder as feedback observation. The present work focuses on the efficiency and robustness of the identified control strategy and introduces a novel algorithm (S-PPO-CMA) to optimise the sensor layout. An energy-efficient control strategy reducing drag by 18.4% at Reynolds number 120 is obtained. This control policy is shown to be robust both to the Reynolds number in the range [100,216] and to measurement noise, enduring signal to noise ratios as low as 0.2 with negligible impact on performance. Along with a systematic study on sensor number and location, the proposed sparsity-seeking algorithm has achieved a successful optimisation to a reduced 5-sensor layout while keeping state-of-the-art performance. These results highlight the interesting possibilities of reinforcement learning for active flow control and pave the way to efficient, robust and practical implementations of these control techniques in experimental or industrial systems.