Nonparametric Operator-Regularized Covariance Function Estimation for Functional Data

TL;DR

This paper introduces a novel nonparametric covariance function estimator for functional data analysis that guarantees positive semi-definiteness, offers a closed-form eigen-decomposition, and achieves low-rank approximation through trace-norm regularization, with proven efficiency and practical success.

Contribution

It develops a new spectral regularization-based covariance estimator with a finite-dimensional representation and automatic positive semi-definiteness, advancing FDA methodology.

Findings

Estimator is positive semi-definite without extra steps.

Trace-norm regularization yields low-rank covariance estimates.

Algorithm demonstrates fast convergence and strong empirical performance.

Abstract

In functional data analysis (FDA), covariance function is fundamental not only as a critical quantity for understanding elementary aspects of functional data but also as an indispensable ingredient for many advanced FDA methods. This paper develops a new class of nonparametric covariance function estimators in terms of various spectral regularizations of an operator associated with a reproducing kernel Hilbert space. Despite their nonparametric nature, the covariance estimators are automatically positive semi-definite without any additional modification steps. An unconventional representer theorem is established to provide a finite dimensional representation for this class of covariance estimators, which leads to a closed-form expression of the corresponding $L^2$ eigen-decomposition. Trace-norm regularization is particularly studied to further achieve a low-rank representation, another desirable property which leads to dimension reduction and is often needed in advanced FDA approaches. An efficient algorithm is developed based on the accelerated proximal gradient method. This resulted estimator is shown to enjoy an excellent rate of convergence under both fixed and random designs. The outstanding practical performance of the trace-norm-regularized covariance estimator is demonstrated by a simulation study and the analysis of a traffic dataset.