An approach of deep reinforcement learning for maximizing the net present value of stochastic projects

TL;DR

This paper presents a novel deep reinforcement learning approach using Double Deep Q-Networks to optimize the net present value of stochastic projects with uncertain durations and cash flows, outperforming traditional strategies.

Contribution

It introduces a DDQN-based method for project optimization formulated as an MDP, demonstrating improved performance and robustness over existing strategies in uncertain environments.

Findings

DDQN outperforms traditional strategies in maximizing expected NPV.

Dual-network architecture reduces overestimation of action values.

Target network enhances training convergence and robustness.

Abstract

This paper investigates a project with stochastic activity durations and cash flows under discrete scenarios, where activities must satisfy precedence constraints generating cash inflows and outflows. The objective is to maximize expected net present value (NPV) by accelerating inflows and deferring outflows. We formulate the problem as a discrete-time Markov Decision Process (MDP) and propose a Double Deep Q-Network (DDQN) approach. Comparative experiments demonstrate that DDQN outperforms traditional rigid and dynamic strategies, particularly in large-scale or highly uncertain environments, exhibiting superior computational capability, policy reliability, and adaptability. Ablation studies further reveal that the dual-network architecture mitigates overestimation of action values, while the target network substantially improves training convergence and robustness. These results indicate that DDQN not only achieves higher expected NPV in complex project optimization but also provides a reliable framework for stable and effective policy implementation.