Deep Reinforcement Learning for Real-Time Optimization of Pumps in Water Distribution Systems

TL;DR

This paper demonstrates that deep reinforcement learning can effectively control water distribution pumps in real-time, offering high efficiency and speed compared to traditional optimization methods, even with only measurement data.

Contribution

The novel contribution is applying a dueling deep Q-network to real-time pump control in water systems, enabling faster and more efficient operation based solely on measurement data.

Findings

DRL agent achieves over 0.98 efficiency relative to baselines.

DRL speeds up control decisions by approximately 2 times.

Agent outperforms conventional optimization in simulated environments.

Abstract

Real-time control of pumps can be an infeasible task in water distribution systems (WDSs) because the calculation to find the optimal pump speeds is resource-intensive. The computational need cannot be lowered even with the capabilities of smart water networks when conventional optimization techniques are used. Deep reinforcement learning (DRL) is presented here as a controller of pumps in two WDSs. An agent based on a dueling deep q-network is trained to maintain the pump speeds based on instantaneous nodal pressure data. General optimization techniques (e.g., Nelder-Mead method, differential evolution) serve as baselines. The total efficiency achieved by the DRL agent compared to the best performing baseline is above 0.98, whereas the speedup is around 2x compared to that. The main contribution of the presented approach is that the agent can run the pumps in real-time because it depends only on measurement data. If the WDS is replaced with a hydraulic simulation, the agent still outperforms conventional techniques in search speed.