Optimization meets Machine Learning: An Exact Algorithm for Semi-Supervised Support Vector Machines

TL;DR

This paper introduces an exact branch-and-cut algorithm for semi-supervised support vector machines (S3VMs) using semidefinite programming relaxations, significantly improving solution bounds and scalability over previous methods.

Contribution

It develops a novel branch-and-cut approach with SDP relaxations and optimality-based bound tightening for S3VMs, enabling exact solutions for larger datasets.

Findings

Algorithm solves instances 10 times larger than previous methods.

SDP relaxations provide stronger bounds than existing approaches.

Computational results demonstrate high efficiency and scalability.

Abstract

Support vector machines (SVMs) are well-studied supervised learning models for binary classification. In many applications, large amounts of samples can be cheaply and easily obtained. What is often a costly and error-prone process is to manually label these instances. Semi-supervised support vector machines (S3VMs) extend the well-known SVM classifiers to the semi-supervised approach, aiming at maximizing the margin between samples in the presence of unlabeled data. By leveraging both labeled and unlabeled data, S3VMs attempt to achieve better accuracy and robustness compared to traditional SVMs. Unfortunately, the resulting optimization problem is non-convex and hence difficult to solve exactly. In this paper, we present a new branch-and-cut approach for S3VMs using semidefinite programming (SDP) relaxations. We apply optimality-based bound tightening to bound the feasible set. Box constraints allow us to include valid inequalities, strengthening the lower bound. The resulting SDP relaxation provides bounds significantly stronger than the ones available in the literature. For the upper bound, instead, we define a local search exploiting the solution of the SDP relaxation. Computational results highlight the efficiency of the algorithm, showing its capability to solve instances with a number of data points 10 times larger than the ones solved in the literature.