Feature space approximation for kernel-based supervised learning

TL;DR

This paper introduces a novel feature space approximation method for kernel-based supervised learning that reduces data size, enhances computational efficiency, and acts as a regularizer, improving model generalization across various applications.

Contribution

The paper presents a new approximation technique for high-dimensional feature vectors that reduces data size and computational costs while improving generalization in supervised learning.

Findings

Significant reduction in storage and computation compared to full data methods

Improved generalization of learned functions due to regularization effect

Effective application across diverse domains like image recognition and time series analysis

Abstract

We propose a method for the approximation of high- or even infinite-dimensional feature vectors, which play an important role in supervised learning. The goal is to reduce the size of the training data, resulting in lower storage consumption and computational complexity. Furthermore, the method can be regarded as a regularization technique, which improves the generalizability of learned target functions. We demonstrate significant improvements in comparison to the computation of data-driven predictions involving the full training data set. The method is applied to classification and regression problems from different application areas such as image recognition, system identification, and oceanographic time series analysis.