SMOG: Scalable Meta-Learning for Multi-Objective Bayesian Optimization

TL;DR

SMOG introduces a scalable meta-learning model using multi-output Gaussian processes to enhance multi-objective Bayesian optimization, leveraging historical data for improved efficiency.

Contribution

The paper presents SMOG, a novel modular meta-learning approach that explicitly models correlations between objectives and scales linearly with meta-tasks.

Findings

SMOG achieves strong data efficiency across benchmarks.

It supports hierarchical, parallel training with linear scaling.

SMOG integrates seamlessly with existing Bayesian optimization methods.

Abstract

Multi-objective optimization aims to solve problems with competing objectives. Evaluating such problems is often slow or expensive, limiting the budget of evaluations. In many applications, historical data from related optimization tasks is available and can be leveraged via meta-learning to accelerate optimization. Bayesian optimization, as a promising technique for expensive black-box problems, has been extended independently to meta-learning and multi-objective optimization, but methods that simultaneously address both settings remain largely unexplored. We propose SMOG-a scalable and modular meta-learning model based on a multi-output Gaussian process-that explicitly learns correlations between objectives. SMOG builds a structured joint Gaussian process prior across meta- and target tasks and, after conditioning on metadata, yields a closed-form prior for the target task. This construction propagates metadata uncertainty into the target surrogate in a principled way. SMOG supports hierarchical, parallel training, achieving linear scaling with the number of meta-tasks. The resulting surrogate integrates seamlessly with standard multi-objective Bayesian optimization acquisition functions. We demonstrate that our method is consistently competitive, delivering strong data efficiency across representative benchmarks and applications.