Experience in Engineering Complex Systems: Active Preference Learning with Multiple Outcomes and Certainty Levels

TL;DR

This paper extends active preference learning algorithms to incorporate richer outcome information, such as certainty levels and multiple outcomes, improving optimization in black-box problems involving human preferences.

Contribution

The paper introduces an enhanced active preference learning algorithm that exploits additional outcome information like Likert scales and multiple outcomes for better optimization.

Findings

Improved optimization performance on benchmark functions.

Effective utilization of outcome certainty levels.

Enhanced handling of multiple outcomes per preference query.

Abstract

Black-box optimization refers to the optimization problem whose objective function and/or constraint sets are either unknown, inaccessible, or non-existent. In many applications, especially with the involvement of humans, the only way to access the optimization problem is through performing physical experiments with the available outcomes being the preference of one candidate with respect to one or many others. Accordingly, the algorithm so-called Active Preference Learning has been developed to exploit this specific information in constructing a surrogate function based on standard radial basis functions, and then forming an easy-to-solve acquisition function which repetitively suggests new decision vectors to search for the optimal solution. Based on this idea, our approach aims to extend the algorithm in such a way that can exploit further information effectively, which can be obtained in reality such as: 5-point Likert type scale for the outcomes of the preference query (i.e., the preference can be described in not only "this is better than that" but also "this is much better than that" level), or multiple outcomes for a single preference query with possible additive information on how certain the outcomes are. The validation of the proposed algorithm is done through some standard benchmark functions, showing a promising improvement with respect to the state-of-the-art algorithm.