Enhancing Pattern Classification in Support Vector Machines through Matrix Formulation

TL;DR

This paper introduces a matrix formulation for Support Vector Machines that improves efficiency and flexibility in multiclass and multilabel classification, enabling better handling of complex learning challenges.

Contribution

The paper presents a novel matrix-based SVM formulation and an accelerated gradient descent method, enhancing computational efficiency and providing new insights for multilabel learning models.

Findings

Matrix SVM achieves superior time efficiency.

Similar classification accuracy to Binary Relevance SVM.

Unveils advantages of matrix formulation over traditional vector-based methods.

Abstract

Support Vector Machines (SVM) have gathered significant acclaim as classifiers due to their successful implementation of Statistical Learning Theory. However, in the context of multiclass and multilabel settings, the reliance on vector-based formulations in existing SVM-based models poses limitations regarding flexibility and ease of incorporating additional terms to handle specific challenges. To overcome these limitations, our research paper focuses on introducing a matrix formulation for SVM that effectively addresses these constraints. By employing the Accelerated Gradient Descent method in the dual, we notably enhance the efficiency of solving the Matrix-SVM problem. Experimental evaluations on multilabel and multiclass datasets demonstrate that Matrix SVM achieves superior time efficacy while delivering similar results to Binary Relevance SVM. Moreover, our matrix formulation unveils crucial insights and advantages that may not be readily apparent in traditional vector-based notations. We emphasize that numerous multilabel models can be viewed as extensions of SVM, with customised modifications to meet specific requirements. The matrix formulation presented in this paper establishes a solid foundation for developing more sophisticated models capable of effectively addressing the distinctive challenges encountered in multilabel learning.