Statistical learnability of smooth boundaries via pairwise binary classification with deep ReLU networks

TL;DR

This paper investigates the theoretical learnability of smooth boundaries using pairwise binary classification with deep ReLU networks, addressing non-identifiability issues and establishing conditions for successful learning.

Contribution

It introduces a new proof method using localization arguments to demonstrate the learnability of ordered smooth boundaries with deep ReLU networks in a pairwise setting.

Findings

Some ordered multiple smooth boundaries are learnable with deep ReLU networks.

The non-identifiability of boundary order is a key challenge addressed in the analysis.

Localization techniques are effective in proving learnability in this context.

Abstract

The topic of nonparametric estimation of smooth boundaries is extensively studied in the conventional setting where pairs of single covariate and response variable are observed. However, this traditional setting often suffers from the cost of data collection. Recent years have witnessed the consistent development of learning algorithms for binary classification problems where one can instead observe paired covariates and binary variable representing the statistical relationship between the covariates. In this work, we theoretically study the learnability of ordered multiple smooth boundaries under a pairwise binary classification setting. One of the challenging problems is the non-identifiability issue on the order of smooth subsets, which yields the gap between the generalizability and the learnability of smooth boundaries in the pairwise binary classification setting. To deal with the challenges due to this non-identifiability directly, we develop a proof method using a localization argument of the given vector-valued function class. Consequently, we prove that some ordered multiple smooth boundaries are learnable via a pairwise binary classification algorithm defined with a localized class of deep ReLU networks.