Finite Sample Prediction and Recovery Bounds for Ordinal Embedding

TL;DR

This paper develops new theoretical bounds and algorithms for ordinal embedding in noisy settings, improving understanding of prediction accuracy and embedding quality from human judgment data.

Contribution

It introduces novel prediction error bounds, analyzes the Maximum Likelihood estimator under known noise, and proposes two new algorithms for ordinal embedding.

Findings

Derived bounds for prediction error in noisy ordinal embedding

Established a relationship between prediction errors and embedding accuracy

Proposed and evaluated two new ordinal embedding algorithms

Abstract

The goal of ordinal embedding is to represent items as points in a low-dimensional Euclidean space given a set of constraints in the form of distance comparisons like "item $i$ is closer to item $j$ than item $k$". Ordinal constraints like this often come from human judgments. To account for errors and variation in judgments, we consider the noisy situation in which the given constraints are independently corrupted by reversing the correct constraint with some probability. This paper makes several new contributions to this problem. First, we derive prediction error bounds for ordinal embedding with noise by exploiting the fact that the rank of a distance matrix of points in $\mathbb{R}^d$ is at most $d+2$. These bounds characterize how well a learned embedding predicts new comparative judgments. Second, we investigate the special case of a known noise model and study the Maximum Likelihood estimator. Third, knowledge of the noise model enables us to relate prediction errors to embedding accuracy. This relationship is highly non-trivial since we show that the linear map corresponding to distance comparisons is non-invertible, but there exists a nonlinear map that is invertible. Fourth, two new algorithms for ordinal embedding are proposed and evaluated in experiments.