Model Predictive and Reinforcement Learning Methods for Active Flow Control of an Airfoil with Dual-point Excitation of Plasma Actuators

TL;DR

This paper compares Model Predictive Control and Reinforcement Learning techniques for active flow control over an airfoil, demonstrating RL's faster adaptation and comparable effectiveness in flow separation management.

Contribution

It introduces a novel comparison between RL and MPC for plasma actuator-based flow control, highlighting RL's rapid adaptation capabilities in real-time aerodynamic applications.

Findings

RL optimized excitation frequency achieving Cl=1.62 in under 2.5 seconds.

Adaptive MPC achieved Cl=1.60 at 110 Hz but struggled near physical limits.

RL methods outperform MPC in dynamic, real-time flow control scenarios.

Abstract

This study investigates the effectiveness of Model Predictive Control (MPC) and Reinforcement Learning (RL) for active flow control over a NACA 4412 airfoil near static stall at Reynolds number 4*10^5. By systematically evaluating these strategies, the research addresses a critical gap in optimizing excitation frequency and improving response time in flow control. The work contributes to understanding RL adaptability and performance versus MPC in aerodynamic flow separation control. Numerical simulations of the Reynolds Averaged Navier-Stokes equations with the Scale-Adaptive Simulation turbulence model are used. Dielectric Barrier Discharge plasma actuators in dual-point excitation mode control flow separation. The study evaluates adaptive MPC, temporal difference RL (TDRL), and deep Q-learning (DQL) for optimizing excitation frequency and expediting stabilization. An integrated signal processing DQL approach is also examined. Adaptive MPC achieved Cl = 1.60 at 110 Hz but struggled near physical limits. RL optimized excitation frequencies, reaching Cl = 1.62 in under 2.5 s at 100 or 200 Hz. The study presents a novel RL - MPC comparison for active flow control with DBD actuators, contrasting with prior work focusing on MPC or RL alone. Using an online learning framework, RL methods dynamically adapt to real-time conditions. Evaluating adaptive MPC and RL together in this setup yields new insights into comparative performance in dynamic environments.