How to craft a deep reinforcement learning policy for wind farm flow control

TL;DR

This paper introduces a novel deep reinforcement learning approach using graph attention networks for wind farm wake steering, significantly improving energy output and robustness in variable wind conditions.

Contribution

It presents the first deep RL-based wake steering controller that generalizes across time-varying wind conditions using a new architecture and training strategy.

Findings

Requires 10x fewer training steps than previous neural networks.

Achieves up to 14% increase in energy production.

Demonstrates robust performance in steady-state simulations.

Abstract

Within wind farms, wake effects between turbines can significantly reduce overall energy production. Wind farm flow control encompasses methods designed to mitigate these effects through coordinated turbine control. Wake steering, for example, consists in intentionally misaligning certain turbines with the wind to optimize airflow and increase power output. However, designing a robust wake steering controller remains challenging, and existing machine learning approaches are limited to quasi-static wind conditions or small wind farms. This work presents a new deep reinforcement learning methodology to develop a wake steering policy that overcomes these limitations. Our approach introduces a novel architecture that combines graph attention networks and multi-head self-attention blocks, alongside a novel reward function and training strategy. The resulting model computes the yaw angles of each turbine, optimizing energy production in time-varying wind conditions. An empirical study conducted on steady-state, low-fidelity simulation, shows that our model requires approximately 10 times fewer training steps than a fully connected neural network and achieves more robust performance compared to a strong optimization baseline, increasing energy production by up to 14 %. To the best of our knowledge, this is the first deep reinforcement learning-based wake steering controller to generalize effectively across any time-varying wind conditions in a low-fidelity, steady-state numerical simulation setting.