Efficient reinforcement learning with partially observable for fluid flow control

TL;DR

This paper introduces a new reinforcement learning algorithm tailored for fluid flow control under partial observability, improving stability and efficiency with fewer observables by incorporating policy parameters into the value function.

Contribution

The authors develop a novel RL algorithm that accounts for partial observability in fluid control, enhancing stability and efficiency over existing methods.

Findings

More stable and efficient than existing RL algorithms

Effective with a small number of observables

Applicable to active flow control problems

Abstract

Despite the low dimensionalities of dissipative viscous fluids, reinforcement learning (RL) requires many observables in fluid control problems. This is because the observables are assumed to follow a policy-independent Markov decision process in the RL framework. By including policy parameters as arguments of a value function, we construct a consistent algorithm with partially observable condition. Using typical examples of active flow control, we show that our algorithm is more stable and efficient than the existing RL algorithms, even under a small number of observables.