On the Statistical Efficiency of Compositional Nonparametric Prediction

TL;DR

This paper introduces a compositional nonparametric prediction method using labeled binary trees, providing theoretical sample complexity bounds and validating them with a greedy regression algorithm on synthetic data.

Contribution

It presents a novel compositional nonparametric model with theoretical sample complexity bounds and a greedy algorithm for regression validation.

Findings

Sample complexity bounds are established for tree recovery.

A greedy regression algorithm effectively validates theoretical results.

Synthetic experiments demonstrate the method's practical viability.

Abstract

In this paper, we propose a compositional nonparametric method in which a model is expressed as a labeled binary tree of $2k+1$ nodes, where each node is either a summation, a multiplication, or the application of one of the $q$ basis functions to one of the $p$ covariates. We show that in order to recover a labeled binary tree from a given dataset, the sufficient number of samples is $O(k\log(pq)+\log(k!))$, and the necessary number of samples is $\Omega(k\log (pq)-\log(k!))$. We further propose a greedy algorithm for regression in order to validate our theoretical findings through synthetic experiments.