Ordinal Regression via Binary Preference vs Simple Regression: Statistical and Experimental Perspectives

TL;DR

This paper compares ordinal regression via binary preference with simple regression, demonstrating through analysis and experiments that preference-based methods can outperform traditional regression in predicting subjective scores.

Contribution

It introduces a rigorous framework showing how reformulating regression as preference tests improves performance and generalizes ORARS to other regression tasks.

Findings

Preference-based regression outperforms simple regression in experiments

Reformulating regression as preference tests enhances accuracy

Conditions for proper application of ORARS are identified

Abstract

Ordinal regression with anchored reference samples (ORARS) has been proposed for predicting the subjective Mean Opinion Score (MOS) of input stimuli automatically. The ORARS addresses the MOS prediction problem by pairing a test sample with each of the pre-scored anchored reference samples. A trained binary classifier is then used to predict which sample, test or anchor, is better statistically. Posteriors of the binary preference decision are then used to predict the MOS of the test sample. In this paper, rigorous framework, analysis, and experiments to demonstrate that ORARS are advantageous over simple regressions are presented. The contributions of this work are: 1) Show that traditional regression can be reformulated into multiple preference tests to yield a better performance, which is confirmed with simulations experimentally; 2) Generalize ORARS to other regression problems and verify its effectiveness; 3) Provide some prerequisite conditions which can insure proper application of ORARS.