Multi-Objective Reinforcement Learning-based Approach for Pressurized Water Reactor Optimization

TL;DR

The paper introduces PEARL, a reinforcement learning method that efficiently solves multi-objective optimization problems in engineering, demonstrated on pressurized water reactor core loading, outperforming traditional approaches in finding Pareto fronts.

Contribution

PEARL is a novel single-policy reinforcement learning approach for multi-objective optimization, eliminating the need for multiple neural networks and effectively handling constraints in complex engineering problems.

Findings

PEARL outperforms classical methods in benchmark tests.

PEARL efficiently finds Pareto fronts in PWR core optimization.

PEARL-NdS variant demonstrates superior performance across metrics.

Abstract

A novel method, the Pareto Envelope Augmented with Reinforcement Learning (PEARL), has been developed to address the challenges posed by multi-objective problems, particularly in the field of engineering where the evaluation of candidate solutions can be time-consuming. PEARL distinguishes itself from traditional policy-based multi-objective Reinforcement Learning methods by learning a single policy, eliminating the need for multiple neural networks to independently solve simpler sub-problems. Several versions inspired from deep learning and evolutionary techniques have been crafted, catering to both unconstrained and constrained problem domains. Curriculum Learning is harnessed to effectively manage constraints in these versions. PEARL's performance is first evaluated on classical multi-objective benchmarks. Additionally, it is tested on two practical PWR core Loading Pattern optimization problems to showcase its real-world applicability. The first problem involves optimizing the Cycle length and the rod-integrated peaking factor as the primary objectives, while the second problem incorporates the mean average enrichment as an additional objective. Furthermore, PEARL addresses three types of constraints related to boron concentration, peak pin burnup, and peak pin power. The results are systematically compared against conventional approaches. Notably, PEARL, specifically the PEARL-NdS variant, efficiently uncovers a Pareto front without necessitating additional efforts from the algorithm designer, as opposed to a single optimization with scaled objectives. It also outperforms the classical approach across multiple performance metrics, including the Hyper-volume.