Gradient-based Quadratic Multiform Separation

TL;DR

This paper explores Quadratic Multiform Separation (QMS), a novel classification method, and proposes a gradient-based optimization approach using Adam to improve its performance, showing it rivals leading machine learning algorithms.

Contribution

It introduces a gradient-based optimization technique for QMS and provides insights into hyperparameter tuning, enhancing its practical applicability.

Findings

QMS achieves accuracy comparable to existing classifiers.

The gradient-based approach effectively optimizes QMS loss.

QMS performance is close to top gradient boosting methods.

Abstract

Classification as a supervised learning concept is an important content in machine learning. It aims at categorizing a set of data into classes. There are several commonly-used classification methods nowadays such as k-nearest neighbors, random forest, and support vector machine. Each of them has its own pros and cons, and none of them is invincible for all kinds of problems. In this thesis, we focus on Quadratic Multiform Separation (QMS), a classification method recently proposed by Michael Fan et al. (2019). Its fresh concept, rich mathematical structure, and innovative definition of loss function set it apart from the existing classification methods. Inspired by QMS, we propose utilizing a gradient-based optimization method, Adam, to obtain a classifier that minimizes the QMS-specific loss function. In addition, we provide suggestions regarding model tuning through explorations of the relationships between hyperparameters and accuracies. Our empirical result shows that QMS performs as good as most classification methods in terms of accuracy. Its superior performance is almost comparable to those of gradient boosting algorithms that win massive machine learning competitions.