Fast Distribution To Real Regression

TL;DR

This paper introduces the Double-Basis estimator for distribution-to-real regression, achieving fast convergence rates and computational efficiency independent of data size, addressing scalability issues of previous methods.

Contribution

The paper proposes the Double-Basis estimator, which reduces computational complexity and maintains fast convergence rates for distribution-to-real regression.

Findings

Double-Basis estimator has complexity independent of data size N.

It achieves a fast convergence rate for a broad class of functions.

The method addresses scalability issues of previous Kernel-Kernel estimators.

Abstract

We study the problem of distribution to real-value regression, where one aims to regress a mapping $f$ that takes in a distribution input covariate $P\in \mathcal{I}$ (for a non-parametric family of distributions $\mathcal{I}$) and outputs a real-valued response $Y=f(P) + \epsilon$. This setting was recently studied, and a "Kernel-Kernel" estimator was introduced and shown to have a polynomial rate of convergence. However, evaluating a new prediction with the Kernel-Kernel estimator scales as $\Omega(N)$. This causes the difficult situation where a large amount of data may be necessary for a low estimation risk, but the computation cost of estimation becomes infeasible when the data-set is too large. To this end, we propose the Double-Basis estimator, which looks to alleviate this big data problem in two ways: first, the Double-Basis estimator is shown to have a computation complexity that is independent of the number of of instances $N$ when evaluating new predictions after training; secondly, the Double-Basis estimator is shown to have a fast rate of convergence for a general class of mappings $f\in\mathcal{F}$.