The Effective Number of Parameters in Kernel Density Estimation

TL;DR

This paper introduces the first oracle-based formula for the effective degrees of freedom in kernel density estimation, using orthogonal polynomial expansions and kernel sensitivity analysis, supported by theoretical and numerical validation.

Contribution

It proposes a novel, theoretically grounded EDoF formula for KDE based on OPS expansions and kernel sensitivity matrices, filling a long-standing gap in the literature.

Findings

The EDoF formula is oracle-based and theoretically justified.

Asymptotic properties of the empirical EDoF are derived.

Numerical results confirm the theoretical predictions.

Abstract

The quest for a formula that satisfactorily measures the effective degrees of freedom in kernel density estimation (KDE) is a long standing problem with few solutions. Starting from the orthogonal polynomial sequence (OPS) expansion for the ratio of the empirical to the oracle density, we show how convolution with the kernel leads to a new OPS with respect to which one may express the resulting KDE. The expansion coefficients of the two OPS systems can then be related via a kernel sensitivity matrix, and this then naturally leads to a definition of effective parameters by taking the trace of a symmetrized positive semi-definite normalized version. The resulting effective degrees of freedom (EDoF) formula is an oracle-based quantity; the first ever proposed in the literature. Asymptotic properties of the empirical EDoF are worked out through influence functions. Numerical investigations confirm the theoretical insights.