Adaptive Conditional Forest Sampling for Spectral Risk Optimisation under Decision-Dependent Uncertainty

TL;DR

This paper introduces ACFS, a novel simulation-optimisation framework that effectively minimizes spectral risk under decision-dependent uncertainty by combining advanced decision-conditional distribution approximation, exploration, and reranking techniques.

Contribution

The paper presents ACFS, a new four-phase framework integrating Generalised Random Forests, CEM-guided exploration, focused augmentation, and reranking, improving spectral risk minimization under decision-dependent uncertainty.

Findings

ACFS achieves the lowest median oracle spectral risk across benchmarks.

ACFS reduces run-to-run dispersion, improving reliability.

ACFS outperforms existing methods like GP-BO, CEM-SO, SGD-CVaR, and KDE-SO.

Abstract

Minimising a spectral risk objective, defined as a convex combination of expected cost and Conditional Value-at-Risk (CVaR), is challenging when the uncertainty distribution is decision-dependent, making both surrogate modelling and simulation-based ranking sensitive to tail estimation error. We propose Adaptive Conditional Forest Sampling (ACFS), a four-phase simulation-optimisation framework that integrates Generalised Random Forests for decision-conditional distribution approximation, CEM-guided global exploration, rank-weighted focused augmentation, and surrogate-to-oracle two-stage reranking before multi-start gradient-based refinement. We evaluate ACFS on two structurally distinct data-generating processes: a decision-dependent Student-t copula and a Gaussian copula with log-normal marginals, across three penalty-weight configurations and 100 replications per setting. ACFS achieves the lowest median oracle spectral risk on the second benchmark in every configuration, with median gaps over GP-BO ranging from 6.0% to 20.0%. On the first benchmark, ACFS and GP-BO are statistically indistinguishable in median objective, but ACFS reduces cross-replication dispersion by approximately 1.8 to 1.9 times on the first benchmark and 1.7 to 2.0 times on the second, indicating materially improved run-to-run reliability. ACFS also outperforms CEM-SO, SGD-CVaR, and KDE-SO in nearly all settings, while ablation and sensitivity analyses support the contribution and robustness of the proposed design.