A Comparison of the Triangle Algorithm and SMO for Solving the Hard Margin Problem

TL;DR

This paper compares the Triangle Algorithm and SMO for solving the hard-margin SVM problem, demonstrating that the Triangle Algorithm outperforms SMO in large-scale experiments involving high-dimensional data.

Contribution

It introduces the Triangle Algorithm as an alternative to SMO for hard-margin SVMs and provides empirical evidence of its superior performance on large datasets.

Findings

Triangle Algorithm outperforms SMO in high-dimensional, large-scale datasets

Experimental results include up to 5000 points per set in 10,000 dimensions

Triangle Algorithm effectively tests convex hull intersection and finds supporting hyperplanes

Abstract

In this article we consider the problem of testing, for two finite sets of points in the Euclidean space, if their convex hulls are disjoint and computing an optimal supporting hyperplane if so. This is a fundamental problem of classification in machine learning known as the hard-margin SVM. The problem can be formulated as a quadratic programming problem. The SMO algorithm is the current state of art algorithm for solving it, but it does not answer the question of separability. An alternative to solving both problems is the Triangle Algorithm, a geometrically inspired algorithm, initially described for the convex hull membership problem, a fundamental problem in linear programming. First, we describe the experimental performance of the Triangle Algorithm for testing the intersection of two convex hulls. Next, we compare the performance of Triangle Algorithm with SMO for finding the optimal supporting hyperplane. Based on experimental results ranging up to 5000 points in each set in dimensions up to 10000, the Triangle Algorithm outperforms SMO.