RBF Kernel Parameter Formula for Data Classification Methods

TL;DR

This paper introduces an analytical formula for selecting the RBF kernel parameter, reducing computational costs and improving robustness in kernel-based classification methods.

Contribution

An efficient, analytical approach for RBF kernel parameter selection that enhances performance and reduces computational expense in SVM and subspace classification.

Findings

The formula effectively optimizes class separation in feature space.

Validation shows improved efficiency in SVM and POD-based methods.

Method performs well in both binary and multi-class classification.

Abstract

Radial Basis Function (RBF), or Gaussian, kernels are among the most widely used parametric kernels in machine learning, particularly in methods such as Support Vector Machines (SVM) and kernel-based subspace approaches. The kernel parameter $\gamma$ (or $\sigma$ in the Gaussian formulation) must be carefully tuned, as the performance of these methods strongly depends on its value and is highly sensitive to improper selection. In practice, this parameter is typically determined through computationally expensive training procedures, which may also lack robustness. In this paper, we propose an efficient analytical formula for selecting the RBF kernel parameter that significantly reduces the computational cost of RBF-based methods. The proposed approach is derived by optimizing the diameter of mapped classes in the feature space while simultaneously maximizing inter-class feature distances. The detailed formulation is presented, and its efficiency is validated on the widely used SVM algorithm as well as on a Proper Orthogonal Decomposition (POD)-based subspace method for both binary and multi-class classification problems.