Convolutional Spectral Kernel Learning

TL;DR

This paper introduces a convolutional spectral kernel network (CSKN) that enhances non-stationary spectral kernels with deep, hierarchical, and local feature learning capabilities, supported by theoretical generalization bounds and empirical validation.

Contribution

The paper develops a novel CSKN model integrating deep architectures and convolutional filters into spectral kernels, along with theoretical analysis and regularizers to improve learning performance.

Findings

CSKN effectively captures hierarchical and local features.

Theoretical generalization bounds support model robustness.

Experimental results validate the model's effectiveness on real datasets.

Abstract

Recently, non-stationary spectral kernels have drawn much attention, owing to its powerful feature representation ability in revealing long-range correlations and input-dependent characteristics. However, non-stationary spectral kernels are still shallow models, thus they are deficient to learn both hierarchical features and local interdependence. In this paper, to obtain hierarchical and local knowledge, we build an interpretable convolutional spectral kernel network (\texttt{CSKN}) based on the inverse Fourier transform, where we introduce deep architectures and convolutional filters into non-stationary spectral kernel representations. Moreover, based on Rademacher complexity, we derive the generalization error bounds and introduce two regularizers to improve the performance. Combining the regularizers and recent advancements on random initialization, we finally complete the learning framework of \texttt{CSKN}. Extensive experiments results on real-world datasets validate the effectiveness of the learning framework and coincide with our theoretical findings.