Handling Imbalanced Classification Problems With Support Vector Machines via Evolutionary Bilevel Optimization

TL;DR

This paper presents EBCS-SVM, an innovative evolutionary bilevel optimization approach that enhances support vector machines for imbalanced classification by jointly optimizing support vectors and hyperparameters.

Contribution

It introduces a novel bilevel optimization framework combining evolutionary algorithms and SMO to improve SVM performance on imbalanced datasets.

Findings

EBCS-SVM outperforms existing methods on highly imbalanced datasets.

The approach effectively optimizes hyperparameters and support vectors.

Experimental results are statistically validated with Bayesian testing.

Abstract

Support vector machines (SVMs) are popular learning algorithms to deal with binary classification problems. They traditionally assume equal misclassification costs for each class; however, real-world problems may have an uneven class distribution. This article introduces EBCS-SVM: evolutionary bilevel cost-sensitive SVMs. EBCS-SVM handles imbalanced classification problems by simultaneously learning the support vectors and optimizing the SVM hyperparameters, which comprise the kernel parameter and misclassification costs. The resulting optimization problem is a bilevel problem, where the lower level determines the support vectors and the upper level the hyperparameters. This optimization problem is solved using an evolutionary algorithm (EA) at the upper level and sequential minimal optimization (SMO) at the lower level. These two methods work in a nested fashion, that is, the optimal support vectors help guide the search of the hyperparameters, and the lower level is initialized based on previous successful solutions. The proposed method is assessed using 70 datasets of imbalanced classification and compared with several state-of-the-art methods. The experimental results, supported by a Bayesian test, provided evidence of the effectiveness of EBCS-SVM when working with highly imbalanced datasets.