Mixed Integer Linear Programming for Feature Selection in Support Vector Machine

TL;DR

This paper introduces a mixed integer linear programming approach for feature selection in support vector machines, incorporating a feature budget constraint and providing both exact and heuristic solutions, validated on standard datasets.

Contribution

It presents a novel MILP formulation for feature selection in SVMs with a predefined feature limit, including efficient solution methods and empirical validation.

Findings

The MILP model effectively selects features within the budget constraint.

The heuristic method provides near-optimal solutions faster than exact methods.

Validation shows competitive classification performance with classical methods.

Abstract

This work focuses on support vector machine (SVM) with feature selection. A MILP formulation is proposed for the problem. The choice of suitable features to construct the separating hyperplanes has been modelled in this formulation by including a budget constraint that sets in advance a limit on the number of features to be used in the classification process. We propose both an exact and a heuristic procedure to solve this formulation in an efficient way. Finally, the validation of the model is done by checking it with some well-known data sets and comparing it with classical classification methods.