Convex-constrained Sparse Additive Modeling and Its Extensions

TL;DR

This paper introduces a novel sparse additive modeling approach that integrates shape constraints like convexity, enabling flexible estimation of continuous functions without smoothness assumptions, with efficient algorithms and competitive performance.

Contribution

It proposes the sparse difference of convex additive models (SDCAM) that incorporate shape constraints and regularization, advancing high-dimensional nonparametric regression methods.

Findings

Competitive performance on synthetic and real data

Efficient backfitting algorithm with linear complexity

Improved results over existing sparse additive models

Abstract

Sparse additive modeling is a class of effective methods for performing high-dimensional nonparametric regression. In this work we show how shape constraints such as convexity/concavity and their extensions, can be integrated into additive models. The proposed sparse difference of convex additive models (SDCAM) can estimate most continuous functions without any a priori smoothness assumption. Motivated by a characterization of difference of convex functions, our method incorporates a natural regularization functional to avoid overfitting and to reduce model complexity. Computationally, we develop an efficient backfitting algorithm with linear per-iteration complexity. Experiments on both synthetic and real data verify that our method is competitive against state-of-the-art sparse additive models, with improved performance in most scenarios.