RankNEAT: Outperforming Stochastic Gradient Search in Preference Learning Tasks

TL;DR

This paper introduces RankNEAT, an evolutionary algorithm that outperforms traditional gradient-based methods in preference learning tasks, especially in noisy, subjective data scenarios like affective computing.

Contribution

RankNEAT is a novel neuroevolution approach that enhances preference learning by optimizing neural network architecture, reducing overfitting, and improving performance over gradient-based methods.

Findings

RankNEAT outperforms RankNet in most experiments.

Architecture optimization acts as an effective feature selection.

RankNEAT reduces overfitting in preference learning tasks.

Abstract

Stochastic gradient descent (SGD) is a premium optimization method for training neural networks, especially for learning objectively defined labels such as image objects and events. When a neural network is instead faced with subjectively defined labels--such as human demonstrations or annotations--SGD may struggle to explore the deceptive and noisy loss landscapes caused by the inherent bias and subjectivity of humans. While neural networks are often trained via preference learning algorithms in an effort to eliminate such data noise, the de facto training methods rely on gradient descent. Motivated by the lack of empirical studies on the impact of evolutionary search to the training of preference learners, we introduce the RankNEAT algorithm which learns to rank through neuroevolution of augmenting topologies. We test the hypothesis that RankNEAT outperforms traditional gradient-based preference learning within the affective computing domain, in particular predicting annotated player arousal from the game footage of three dissimilar games. RankNEAT yields superior performances compared to the gradient-based preference learner (RankNet) in the majority of experiments since its architecture optimization capacity acts as an efficient feature selection mechanism, thereby, eliminating overfitting. Results suggest that RankNEAT is a viable and highly efficient evolutionary alternative to preference learning.