Submodular Optimization for Efficient Semi-supervised Support Vector Machines

TL;DR

This paper introduces an efficient quadratic programming approximation for semi-supervised SVMs, unifies various SSL models through submodular functions, and demonstrates improved accuracy and computational speed.

Contribution

It presents a novel quadratic programming approximation for S3VM, links SSL models via submodular functions, and employs greedy algorithms for faster optimization.

Findings

Significant reduction in training time compared to existing methods

Improved accuracy in semi-supervised learning tasks

Effective unification of low density separation and graph-based SSL models

Abstract

In this work we present a quadratic programming approximation of the Semi-Supervised Support Vector Machine (S3VM) problem, namely approximate QP-S3VM, that can be efficiently solved using off the shelf optimization packages. We prove that this approximate formulation establishes a relation between the low density separation and the graph-based models of semi-supervised learning (SSL) which is important to develop a unifying framework for semi-supervised learning methods. Furthermore, we propose the novel idea of representing SSL problems as submodular set functions and use efficient submodular optimization algorithms to solve them. Using this new idea we develop a representation of the approximate QP-S3VM as a maximization of a submodular set function which makes it possible to optimize using efficient greedy algorithms. We demonstrate that the proposed methods are accurate and provide significant improvement in time complexity over the state of the art in the literature.