Solving Support Vector Machines in Reproducing Kernel Banach Spaces with Positive Definite Functions

TL;DR

This paper introduces a novel approach to support vector machines using reproducing kernel Banach spaces with positive definite functions, enabling explicit solutions and efficient computation beyond traditional Hilbert space methods.

Contribution

It develops a framework for SVMs in reproducing kernel Banach spaces on nonsymmetric domains, providing explicit solution representations and practical algorithms.

Findings

Explicit dual element representations for SVM solutions

Reproducing kernel Banach spaces embedded into Sobolev spaces

Efficient fixed point algorithms for computing SVM coefficients

Abstract

In this paper we solve support vector machines in reproducing kernel Banach spaces with reproducing kernels defined on nonsymmetric domains instead of the traditional methods in reproducing kernel Hilbert spaces. Using the orthogonality of semi-inner-products, we can obtain the explicit representations of the dual (normalized-duality-mapping) elements of support vector machine solutions. In addition, we can introduce the reproduction property in a generalized native space by Fourier transform techniques such that it becomes a reproducing kernel Banach space, which can be even embedded into Sobolev spaces, and its reproducing kernel is set up by the related positive definite function. The representations of the optimal solutions of support vector machines (regularized empirical risks) in these reproducing kernel Banach spaces are formulated explicitly in terms of positive definite functions, and their finite numbers of coefficients can be computed by fixed point iteration. We also give some typical examples of reproducing kernel Banach spaces induced by Mat\'ern functions (Sobolev splines) so that their support vector machine solutions are well computable as the classical algorithms. Moreover, each of their reproducing bases includes information from multiple training data points. The concept of reproducing kernel Banach spaces offers us a new numerical tool for solving support vector machines.