Quantifying the cost of simultaneous non-parametric approximation of several samples

TL;DR

This paper investigates the feasibility of representing multiple samples with a single regression function using confidence regions, and introduces a measure for the cost of joint approximation in non-parametric regression with Gaussian errors.

Contribution

It proposes a method to determine when multiple samples can be jointly approximated by one function and introduces a measure for the approximation cost based on additional features needed.

Findings

Confidence regions can test joint representability of samples.

Disjoint supports imply non-empty intersection only under shape constraints.

A simple joint approximation function quantifies the approximation cost.

Abstract

We consider the standard non-parametric regression model with Gaussian errors but where the data consist of different samples. The question to be answered is whether the samples can be adequately represented by the same regression function. To do this we define for each sample a universal, honest and non-asymptotic confidence region for the regression function. Any subset of the samples can be represented by the same function if and only if the intersection of the corresponding confidence regions is non-empty. If the empirical supports of the samples are disjoint then the intersection of the confidence regions is always non--empty and a negative answer can only be obtained by placing shape or quantitative smoothness conditions on the joint approximation. Alternatively a simplest joint approximation function can be calculated which gives a measure of the cost of the joint approximation, for example, the number of extra peaks required.