Learning Distributions over Permutations and Rankings with Factorized Representations

TL;DR

This paper introduces a novel permutation learning method using bijective representations that enables unconstrained deep learning, outperforming existing models on benchmarks and allowing flexible trade-offs between expressivity and computational cost.

Contribution

The authors propose a new approach leveraging permutation representations like Lehmer codes, enabling unconstrained deep learning over permutations and unifying various probabilistic models.

Findings

Outperforms current approaches on the jigsaw puzzle benchmark.

Can learn non-trivial permutation distributions even in low expressivity modes.

Traditional models fail to generate valid permutations in minimal expressivity settings.

Abstract

Learning distributions over permutations is a fundamental problem in machine learning, with applications in ranking, combinatorial optimization, structured prediction, and data association. Existing methods rely on mixtures of parametric families or neural networks with expensive variational inference procedures. In this work, we propose a novel approach that leverages alternative representations for permutations, including Lehmer codes, Fisher-Yates draws, and Insertion-Vectors. These representations form a bijection with the symmetric group, allowing for unconstrained learning using conventional deep learning techniques, and can represent any probability distribution over permutations. Our approach enables a trade-off between expressivity of the model family and computational requirements. In the least expressive and most computationally efficient case, our method subsumes previous families of well established probabilistic models over permutations, including Mallow's and the Repeated Insertion Model. Experiments indicate our method significantly outperforms current approaches on the jigsaw puzzle benchmark, a common task for permutation learning. However, we argue this benchmark is limited in its ability to assess learning probability distributions, as the target is a delta distribution (i.e., a single correct solution exists). We therefore propose two additional benchmarks: learning cyclic permutations and re-ranking movies based on user preference. We show that our method learns non-trivial distributions even in the least expressive mode, while traditional models fail to even generate valid permutations in this setting.