Asymptotic Properties of Penalized Spline Estimators in Concave Extended Linear Models: Rates of Convergence

TL;DR

This paper establishes a comprehensive theory on the convergence rates of penalized spline estimators across various models and likelihood types, highlighting the influence of knot number and penalty parameters on asymptotic behavior.

Contribution

It introduces a unified framework for analyzing the asymptotic properties of penalized spline estimators in diverse settings, including generalized regression and density estimation.

Findings

Convergence rates depend on the interplay between knot number and penalty parameter.

The theory applies to multiple models like logistic, Poisson, and density estimation.

Extensions to multi-dimensional spline estimation are also provided.

Abstract

This paper develops a general theory on rates of convergence of penalized spline estimators for function estimation when the likelihood functional is concave in candidate functions, where the likelihood is interpreted in a broad sense that includes conditional likelihood, quasi-likelihood, and pseudo-likelihood. The theory allows all feasible combinations of the spline degree, the penalty order, and the smoothness of the unknown functions. According to this theory, the asymptotic behaviors of the penalized spline estimators depends on interplay between the spline knot number and the penalty parameter. The general theory is applied to obtain results in a variety of contexts, including regression, generalized regression such as logistic regression and Poisson regression, density estimation, conditional hazard function estimation for censored data, quantile regression, diffusion function estimation for a diffusion type process, and estimation of spectral density function of a stationary time series. For multi-dimensional function estimation, the theory (presented in the Supplementary Material) covers both penalized tensor product splines and penalized bivariate splines on triangulations.