ParamBoost: Gradient Boosted Piecewise Cubic Polynomials

TL;DR

ParamBoost is a novel interpretable GAM that uses gradient boosting with cubic polynomials, allowing for flexible shape functions with constraints like monotonicity and convexity, improving performance and interpretability.

Contribution

It introduces a gradient boosting approach for GAMs with shape functions modeled by cubic polynomials, incorporating constraints for enhanced interpretability and application-specific tailoring.

Findings

Unconstrained ParamBoost outperforms existing GAMs on real datasets.

Imposing constraints slightly reduces performance but enhances interpretability.

The model can be tailored to specific interpretability and analysis needs.

Abstract

Generalized Additive Models (GAMs) can be used to create non-linear glass-box (i.e. explicitly interpretable) models, where the predictive function is fully observable over the complete input space. However, glass-box interpretability itself does not allow for the incorporation of expert knowledge from the modeller. In this paper, we present ParamBoost, a novel GAM whose shape functions (i.e. mappings from individual input features to the output) are learnt using a Gradient Boosting algorithm that fits cubic polynomial functions at leaf nodes. ParamBoost incorporates several constraints commonly used in parametric analysis to ensure well-refined shape functions. These constraints include: (i) continuity of the shape functions and their derivatives (up to C2); (ii) monotonicity; (iii) convexity; (iv) feature interaction constraints; and (v) model specification constraints. Empirical results show that the unconstrained ParamBoost model consistently outperforms state-of-the-art GAMs across several real-world datasets. We further demonstrate that modellers can selectively impose required constraints at a modest trade-off in predictive performance, allowing the model to be fully tailored to application-specific interpretability and parametric-analysis requirements.