Winner-takes-all learners are geometry-aware conditional density estimators

TL;DR

This paper demonstrates how Winner-takes-all learners can be effectively used for conditional density estimation by leveraging their geometric properties, providing theoretical advantages and empirical performance on diverse datasets.

Contribution

It introduces a novel approach to use Winner-takes-all learners for density estimation without altering training, connecting geometric properties to improved uncertainty quantification.

Findings

The proposed estimator has theoretical advantages in quantization and density estimation.

It performs competitively on synthetic and real-world datasets, including audio.

The approach leverages geometric properties of Winner-takes-all learners for better uncertainty quantification.

Abstract

Winner-takes-all training is a simple learning paradigm, which handles ambiguous tasks by predicting a set of plausible hypotheses. Recently, a connection was established between Winner-takes-all training and centroidal Voronoi tessellations, showing that, once trained, hypotheses should quantize optimally the shape of the conditional distribution to predict. However, the best use of these hypotheses for uncertainty quantification is still an open question. In this work, we show how to leverage the appealing geometric properties of the Winner-takes-all learners for conditional density estimation, without modifying its original training scheme. We theoretically establish the advantages of our novel estimator both in terms of quantization and density estimation, and we demonstrate its competitiveness on synthetic and real-world datasets, including audio data.