When Less is More: A Story of Failing Bayesian Optimization Due to Additional Expert Knowledge

TL;DR

This paper investigates how adding expert knowledge and data to Bayesian Optimization can sometimes hinder rather than help, emphasizing the importance of simplicity in the optimization goal.

Contribution

It highlights the potential pitfalls of incorporating extra data and knowledge into Bayesian Optimization and provides insights into when such additions may impair performance.

Findings

Adding data and knowledge can impair optimization performance.

Unsuccessful methods were analyzed to understand failure causes.

Simplifying the optimization goal can improve results.

Abstract

The compounding of plastics with recycled material remains a practical challenge, as the properties of the processed material is not as easy to control as with completely new raw materials. For a data scientist, it makes sense to plan the necessary experiments in the development of new compounds using Bayesian Optimization, an optimization approach based on a surrogate model that is known for its data efficiency and is therefore well suited for data obtained from costly experiments. Furthermore, if historical data and expert knowledge are available, their inclusion in the surrogate model is expected to accelerate the convergence of the optimization. In this article, we describe a use case in which the addition of data and knowledge has impaired optimization. We also describe the unsuccessful methods that were used to remedy the problem before we found the reasons for the poor performance and achieved a satisfactory result. We conclude with a lesson learned: additional knowledge and data are only beneficial if they do not complicate the underlying optimization goal.