Robust Bayesian Optimization with Student-t Likelihood

TL;DR

This paper introduces a robust Bayesian optimization method using a Student-t likelihood in Gaussian process models to effectively handle outliers, improving hyperparameter tuning in noisy environments.

Contribution

It proposes a novel GP model with Student-t likelihood for outlier robustness in Bayesian optimization, enhancing performance over traditional Gaussian process models.

Findings

Improved robustness to outliers in hyperparameter tuning.

Effective in both artificial and real-world optimization tasks.

Enhanced exploration efficiency in noisy settings.

Abstract

Bayesian optimization has recently attracted the attention of the automatic machine learning community for its excellent results in hyperparameter tuning. BO is characterized by the sample efficiency with which it can optimize expensive black-box functions. The efficiency is achieved in a similar fashion to the learning to learn methods: surrogate models (typically in the form of Gaussian processes) learn the target function and perform intelligent sampling. This surrogate model can be applied even in the presence of noise; however, as with most regression methods, it is very sensitive to outlier data. This can result in erroneous predictions and, in the case of BO, biased and inefficient exploration. In this work, we present a GP model that is robust to outliers which uses a Student-t likelihood to segregate outliers and robustly conduct Bayesian optimization. We present numerical results evaluating the proposed method in both artificial functions and real problems.