Estimating conditional quantiles with the help of the pinball loss

TL;DR

This paper analyzes the efficiency of the pinball loss in estimating conditional quantiles, providing inequalities and variance bounds that lead to optimal learning rates for support vector machines.

Contribution

It establishes new inequalities and variance bounds for nonparametric pinball loss minimizers, and derives an oracle inequality with min-max optimal rates for SVMs.

Findings

Inequalities describing proximity of approximate minimizers to true quantiles

Variance bounds for empirical risk minimization with pinball loss

Oracle inequality with optimal learning rates for SVMs

Abstract

The so-called pinball loss for estimating conditional quantiles is a well-known tool in both statistics and machine learning. So far, however, only little work has been done to quantify the efficiency of this tool for nonparametric approaches. We fill this gap by establishing inequalities that describe how close approximate pinball risk minimizers are to the corresponding conditional quantile. These inequalities, which hold under mild assumptions on the data-generating distribution, are then used to establish so-called variance bounds, which recently turned out to play an important role in the statistical analysis of (regularized) empirical risk minimization approaches. Finally, we use both types of inequalities to establish an oracle inequality for support vector machines that use the pinball loss. The resulting learning rates are min--max optimal under some standard regularity assumptions on the conditional quantile.