Symmetry: a General Structure in Nonparametric Regression

TL;DR

This paper introduces a general framework for symmetry in nonparametric regression, extending covariate sparsity to more complex symmetries, and demonstrates how exploiting these symmetries can improve estimation rates.

Contribution

It generalizes the concept of covariate sparsity to include various symmetries, providing methods to estimate and leverage unknown symmetries for faster regression estimation.

Findings

Symmetries can be exploited for faster convergence rates.

Explicit symmetrisation operators improve estimator performance.

Finite sample results show practical effectiveness.

Abstract

In this paper we present the framework of symmetry in nonparametric regression. This generalises the framework of covariate sparsity, where the regression function depends only on at most $s < d$ of the covariates, which is a special case of translation symmetry with linear orbits. In general this extends to other types of functions that capture lower dimensional behavior even when these structures are non-linear. We show both that known symmetries of regression functions can be exploited to give similarly faster rates, and that unknown symmetries with Lipschitz actions can be estimated sufficiently quickly to obtain the same rates. This is done by explicit constructions of partial symmetrisation operators that are then applied to usual estimators, and with a two step M-estimator of the maximal symmetry of the regression function. We also demonstrate the finite sample performance of these estimators on synthetic data.