Multiclass Graph-Based Large Margin Classifiers: Unified Approach for Support Vectors and Neural Networks

TL;DR

This paper introduces a unified graph-based large margin classification framework that improves support vector and neural network methods using Gabriel graphs, resulting in better performance and computational efficiency.

Contribution

It presents a novel GG-based classifier, a new neural network architecture, and an efficient GG recomputation algorithm for multiclass classification.

Findings

Outperforms previous GG-based classifiers in experiments.

Statistically comparable to tree-based models.

Achieves low-probability margins and smoother classification contours.

Abstract

While large margin classifiers are originally an outcome of an optimization framework, support vectors (SVs) can be obtained from geometric approaches. This article presents advances in the use of Gabriel graphs (GGs) in binary and multiclass classification problems. For Chipclass, a hyperparameter-less and optimization-less GG-based binary classifier, we discuss how activation functions and support edge (SE)-centered neurons affect the classification, proposing smoother functions and structural SV (SSV)-centered neurons to achieve margins with low probabilities and smoother classification contours. We extend the neural network architecture, which can be trained with backpropagation with a softmax function and a cross-entropy loss, or by solving a system of linear equations. A new subgraph-/distance-based membership function for graph regularization is also proposed, along with a new GG recomputation algorithm that is less computationally expensive than the standard approach. Experimental results with the Friedman test show that our method was better than previous GG-based classifiers and statistically equivalent to tree-based models.