Totally Concave Regression

TL;DR

This paper introduces a new multivariate shape-constrained regression method based on total concavity, balancing flexibility and overfitting, with favorable convergence rates and practical effectiveness demonstrated on real datasets.

Contribution

It proposes a novel total concavity-based regression approach that captures covariate interactions and mitigates high-dimensional overfitting, with theoretical guarantees and empirical validation.

Findings

Convergence rates depend logarithmically on covariate number.

Method effectively captures interactions in high dimensions.

Empirical results show practical applicability on real datasets.

Abstract

Shape constraints in nonparametric regression provide a powerful framework for estimating regression functions under realistic assumptions without tuning parameters. However, most existing methods$\unicode{x2013}$except additive models$\unicode{x2013}$impose too weak restrictions, often leading to overfitting in high dimensions. Conversely, additive models can be too rigid, failing to capture covariate interactions. This paper introduces a novel multivariate shape-constrained regression approach based on total concavity, originally studied by T. Popoviciu. Our method allows interactions while mitigating the curse of dimensionality, with convergence rates that depend only logarithmically on the number of covariates. We characterize and compute the least squares estimator over totally concave functions, derive theoretical guarantees, and demonstrate its practical effectiveness through empirical studies on real-world datasets.