DATELINE: Deep Plackett-Luce Model with Uncertainty Measurements

TL;DR

DATELINE introduces a deep neural network-based Plackett-Luce model that incorporates feature information and uncertainty measurements to improve preference aggregation in various real-world applications.

Contribution

It presents a novel deep learning framework that integrates instance features and dynamic uncertainty weights into the Plackett-Luce model for preference aggregation.

Findings

The model effectively captures feature-dependent preferences.

It accurately measures and incorporates uncertainty in crowd-sourced preferences.

Theoretical guarantees demonstrate robustness of DATELINE.

Abstract

The aggregation of k-ary preferences is a historical and important problem, since it has many real-world applications, such as peer grading, presidential elections and restaurant ranking. Meanwhile, variants of Plackett-Luce model has been applied to aggregate k-ary preferences. However, there are two urgent issues still existing in the current variants. First, most of them ignore feature information. Namely, they consider k-ary preferences instead of instance-dependent k-ary preferences. Second, these variants barely consider the uncertainty in k-ary preferences provided by agnostic crowds. In this paper, we propose Deep plAckeTt-luce modEL wIth uNcertainty mEasurements (DATELINE), which can address both issues simultaneously. To address the first issue, we employ deep neural networks mapping each instance into its ranking score in Plackett-Luce model. Then, we present a weighted Plackett-Luce model to solve the second issue, where the weight is a dynamic uncertainty vector measuring the worker quality. More importantly, we provide theoretical guarantees for DATELINE to justify its robustness.