Optimal-$k$ difference sequence in nonparametric regression

TL;DR

This paper introduces the optimal-$k$ difference sequence for nonparametric regression, improving residual variance estimation by balancing bias and variance better than existing sequences, and demonstrating superior practical performance.

Contribution

It proposes a new difference sequence, the optimal-$k$, which generalizes existing sequences and enhances the bias-variance trade-off in nonparametric regression.

Findings

Optimal-$k$ sequence outperforms traditional sequences in simulations.

Theoretical analysis confirms improved bias-variance balance.

Numerical studies show best practical performance of the new sequence.

Abstract

Difference-based methods have been attracting increasing attention in nonparametric regression, in particular for estimating the residual variance.To implement the estimation, one needs to choose an appropriate difference sequence, mainly between {\em the optimal difference sequence} and {\em the ordinary difference sequence}. The difference sequence selection is a fundamental problem in nonparametric regression, and it remains a controversial issue for over three decades. In this paper, we propose to tackle this challenging issue from a very unique perspective, namely by introducing a new difference sequence called {\em the optimal-$k$ difference sequence}. The new difference sequence not only provides a better balance between the bias-variance trade-off, but also dramatically enlarges the existing family of difference sequences that includes the optimal and ordinary difference sequences as two important special cases. We further demonstrate, by both theoretical and numerical studies, that the optimal-$k$ difference sequence has been pushing the boundaries of our knowledge in difference-based methods in nonparametric regression, and it always performs the best in practical situations.