On kernel methods for covariates that are rankings

TL;DR

This paper explores kernel methods for regression and testing with permutation-valued features, characterizing their properties and demonstrating their effectiveness on survey and ratings data.

Contribution

It introduces and analyzes Kendall and Mallows kernels for permutation features, revealing their spectral properties and universality, and proposes polynomial kernels bridging these methods.

Findings

Mallows kernel is universal and characteristic.

Kendall kernel has limited expressive power due to degeneracy.

Polynomial kernels effectively interpolate between Kendall and Mallows kernels.

Abstract

Permutation-valued features arise in a variety of applications, either in a direct way when preferences are elicited over a collection of items, or an indirect way in which numerical ratings are converted to a ranking. To date, there has been relatively limited study of regression, classification, and testing problems based on permutation-valued features, as opposed to permutation-valued responses. This paper studies the use of reproducing kernel Hilbert space methods for learning from permutation-valued features. These methods embed the rankings into an implicitly defined function space, and allow for efficient estimation of regression and test functions in this richer space. Our first contribution is to characterize both the feature spaces and spectral properties associated with two kernels for rankings, the Kendall and Mallows kernels. Using tools from representation theory, we explain the limited expressive power of the Kendall kernel by characterizing its degenerate spectrum, and in sharp contrast, we prove that Mallows' kernel is universal and characteristic. We also introduce families of polynomial kernels that interpolate between the Kendall (degree one) and Mallows' (infinite degree) kernels. We show the practical effectiveness of our methods via applications to Eurobarometer survey data as well as a Movielens ratings dataset.