Direct Preference-Based Evolutionary Multi-Objective Optimization with Dueling Bandit

TL;DR

This paper introduces a novel preference-based multi-objective evolutionary algorithm that uses human feedback via a dueling bandit approach, eliminating the need for explicit fitness functions and demonstrating effectiveness in practical applications like protein structure prediction.

Contribution

It proposes an innovative interactive framework combining dueling bandits with MOEAs, enabling preference learning directly from human feedback without explicit fitness functions.

Findings

Effective preference learning via dueling bandits

Successful integration into MOEAs for optimization

Practical application in protein structure prediction

Abstract

Optimization problems find widespread use in both single-objective and multi-objective scenarios. In practical applications, users aspire for solutions that converge to the region of interest (ROI) along the Pareto front (PF). While the conventional approach involves approximating a fitness function or an objective function to reflect user preferences, this paper explores an alternative avenue. Specifically, we aim to discover a method that sidesteps the need for calculating the fitness function, relying solely on human feedback. Our proposed approach entails conducting direct preference learning facilitated by an active dueling bandit algorithm. The experimental phase is structured into three sessions. Firstly, we assess the performance of our active dueling bandit algorithm. Secondly, we implement our proposed method within the context of Multi-objective Evolutionary Algorithms (MOEAs). Finally, we deploy our method in a practical problem, specifically in protein structure prediction (PSP). This research presents a novel interactive preference-based MOEA framework that not only addresses the limitations of traditional techniques but also unveils new possibilities for optimization problems.