Challenges in Binary Classification

TL;DR

This paper examines the limitations of SVMs in nonlinear binary classification, proposing a variational framework for optimal classifier design and highlighting open challenges in the field.

Contribution

It introduces a variational problem framework for binary classification, unifies SVM as a special case, and discusses the limitations and open problems in designing optimal classifiers.

Findings

SVM is a special case of the proposed variational framework.

The variational problem has limitations for nonlinear classification.

Designing more appropriate objective functions remains an open challenge.

Abstract

Binary Classification plays an important role in machine learning. For linear classification, SVM is the optimal binary classification method. For nonlinear classification, the SVM algorithm needs to complete the classification task by using the kernel function. Although the SVM algorithm with kernel function is very effective, the selection of kernel function is empirical, which means that the kernel function may not be optimal. Therefore, it is worth studying how to obtain an optimal binary classifier. In this paper, the problem of finding the optimal binary classifier is considered as a variational problem. We design the objective function of this variational problem through the max-min problem of the (Euclidean) distance between two classes. For linear classification, it can be deduced that SVM is a special case of this variational problem framework. For Euclidean distance, it is proved that the proposed variational problem has some limitations for nonlinear classification. Therefore, how to design a more appropriate objective function to find the optimal binary classifier is still an open problem. Further, it's discussed some challenges and problems in finding the optimal classifier.