# A Large Dimensional Analysis of Least Squares Support Vector Machines

## TL;DR

This paper provides a large-dimensional theoretical analysis of LS-SVMs using random matrix theory, showing how their decision functions behave under high-dimensional Gaussian mixture models and applying findings to real datasets.

## Contribution

It offers a novel large-dimensional analysis of LS-SVMs, revealing how kernel functions influence their performance in high-dimensional settings.

## Key findings

- Decision function approximates a normal distribution in high dimensions
- Explicit dependence of mean and variance on kernel function
- Application to MNIST datasets confirms theoretical predictions

## Abstract

In this article, a large dimensional performance analysis of kernel least squares support vector machines (LS-SVMs) is provided under the assumption of a two-class Gaussian mixture model for the input data. Building upon recent advances in random matrix theory, we show, when the dimension of data $p$ and their number $n$ are both large, that the LS-SVM decision function can be well approximated by a normally distributed random variable, the mean and variance of which depend explicitly on a local behavior of the kernel function. This theoretical result is then applied to the MNIST and Fashion-MNIST datasets which, despite their non-Gaussianity, exhibit a convincingly close behavior. Most importantly, our analysis provides a deeper understanding of the mechanism into play in SVM-type methods and in particular of the impact on the choice of the kernel function as well as some of their theoretical limits in separating high dimensional Gaussian vectors.