Neural Bradley-Terry Rating: Quantifying Properties from Comparisons

TL;DR

This paper introduces Neural Bradley-Terry Rating (NBTR), a neural network framework that quantifies properties of unknown items by integrating the Bradley-Terry model, even in asymmetric and unfair environments, demonstrating effective property estimation.

Contribution

The paper presents a novel neural network-based framework that incorporates the Bradley-Terry model for property quantification, extending it to handle asymmetric and unfair scenarios.

Findings

Successfully learns to quantify and estimate properties from comparisons.

Effectively handles asymmetric and unfair environments.

Demonstrates robustness in experimental analysis.

Abstract

Many properties in the real world don't have metrics and can't be numerically observed, making them difficult to learn. To deal with this challenging problem, prior works have primarily focused on estimating those properties by using graded human scores as the target label in the training. Meanwhile, rating algorithms based on the Bradley-Terry model are extensively studied to evaluate the competitiveness of players based on their match history. In this paper, we introduce the Neural Bradley-Terry Rating (NBTR), a novel machine learning framework designed to quantify and evaluate properties of unknown items. Our method seamlessly integrates the Bradley-Terry model into the neural network structure. Moreover, we generalize this architecture further to asymmetric environments with unfairness, a condition more commonly encountered in real-world settings. Through experimental analysis, we demonstrate that NBTR successfully learns to quantify and estimate desired properties.