On the Impact of Monte Carlo Statistical Uncertainty on Surrogate-based Design Optimization

TL;DR

This paper examines how Monte Carlo simulation uncertainty affects the accuracy of surrogate models in multi-objective design optimization, revealing problem-dependent impacts and the importance of multi-fidelity approaches.

Contribution

It provides a systematic analysis of the effects of data uncertainty on surrogate-based optimization across different problems and proposes multi-fidelity strategies for improved results.

Findings

Higher uncertainty distorts Pareto-fronts in some problems

Surrogate models can still approximate Pareto-fronts under noise

Multi-fidelity approaches help balance accuracy and computational cost

Abstract

In multi-objective design tasks, the computational cost increases rapidly when high-fidelity simulations are used to evaluate objective functions. Surrogate models help mitigate this cost by approximating the simulation output, simplifying the design process. However, under high uncertainty, surrogate models trained on noisy data can produce inaccurate predictions, as their performance depends heavily on the quality of training data. This study investigates the impact of data uncertainty on two multi-objective design problems modelled using Monte Carlo transport simulations: a neutron moderator and an ion-to-neutron converter. For each, a grid search was performed using five different tally uncertainty levels to generate training data for neural network surrogate models. These models were then optimized using NSGA-III. The recovered Pareto-fronts were analyzed across uncertainty levels, and the impact of training data quality on optimization outcomes was quantified. Average simulation times were also compared to evaluate the trade-off between accuracy and computational cost. Results show that the influence of simulation uncertainty is strongly problem-dependent. In the neutron moderator case, higher uncertainties led to exaggerated objective sensitivities and distorted Pareto-fronts, reducing normalized hypervolume. In contrast, the ion-to-neutron converter task was less affected--low-fidelity simulations produced results similar to those from high-fidelity data. These findings suggest that a fixed-fidelity approach is not optimal. Surrogate models can still recover the Pareto-front under noisy conditions, and multi-fidelity studies can help identify the appropriate uncertainty level for each problem, enabling better trade-offs between computational efficiency and optimization accuracy.