Multi-Objective Multi-Fidelity Bayesian Optimization with Causal Priors

TL;DR

RESCUE introduces a causal modeling approach to multi-fidelity Bayesian optimization, improving sample efficiency by better understanding the causal relationships between variables, and outperforming existing methods on various real-world tasks.

Contribution

It develops a causal framework for multi-fidelity Bayesian optimization, enabling more effective selection of evaluation points by leveraging causal structures.

Findings

RESCUE outperforms state-of-the-art MFBO methods in synthetic and real-world tasks.

Incorporating causal models enhances sample efficiency in multi-objective optimization.

The causal hypervolume knowledge-gradient improves decision-making in multi-fidelity settings.

Abstract

Multi-fidelity Bayesian optimization (MFBO) accelerates the search for the global optimum of black-box functions by integrating inexpensive, low-fidelity approximations. The central task of an MFBO policy is to balance the cost-efficiency of low-fidelity proxies against their reduced accuracy to ensure effective progression toward the high-fidelity optimum. Existing MFBO methods primarily capture associational dependencies between inputs, fidelities, and objectives, rather than causal mechanisms, and can perform poorly when lower-fidelity proxies are poorly aligned with the target fidelity. We propose RESCUE (REducing Sampling cost with Causal Understanding and Estimation), a multi-objective MFBO method that incorporates causal calculus to systematically address this challenge. RESCUE learns a structural causal model capturing causal relationships between inputs, fidelities, and objectives, and uses it to construct a probabilistic multi-fidelity (MF) surrogate that encodes intervention effects. Exploiting the causal structure, we introduce a causal hypervolume knowledge-gradient acquisition strategy to select input-fidelity pairs that balance expected multi-objective improvement and cost. We show that RESCUE improves sample efficiency over state-of-the-art MF optimization methods on synthetic and real-world problems in robotics, machine learning (AutoML), and healthcare.