Componentwise Least Squares Support Vector Machines

TL;DR

This paper introduces componentwise LS-SVMs for estimating additive models with nonlinear components, providing derivations for classification and regression, and proposing techniques for structure discovery through regularization and validation.

Contribution

It presents a novel formulation of componentwise LS-SVMs for additive models, including methods for sparse component detection and structure discovery.

Findings

Linear equations grow with data points, enabling scalable computation.

Regularization schemes facilitate sparse component identification.

Fusion with validation improves structure detection accuracy.

Abstract

This chapter describes componentwise Least Squares Support Vector Machines (LS-SVMs) for the estimation of additive models consisting of a sum of nonlinear components. The primal-dual derivations characterizing LS-SVMs for the estimation of the additive model result in a single set of linear equations with size growing in the number of data-points. The derivation is elaborated for the classification as well as the regression case. Furthermore, different techniques are proposed to discover structure in the data by looking for sparse components in the model based on dedicated regularization schemes on the one hand and fusion of the componentwise LS-SVMs training with a validation criterion on the other hand. (keywords: LS-SVMs, additive models, regularization, structure detection)