Parallel Algorithms for Structured Sparse Support Vector Machines: Application in Music Genre Classification

TL;DR

This paper introduces a scalable parallel algorithm for structured sparse support vector machines, applicable to large-scale distributed data, with a focus on music genre classification, demonstrating efficiency and robustness through theoretical and experimental validation.

Contribution

It develops a unified, scalable optimization framework and a distributed ADMM algorithm for SS-SVMs, extending to non-convex regularizers and applying to music information retrieval.

Findings

Algorithm is scalable and independent of regularization choices.

Experimental results show high reliability and efficiency.

Framework applicable to various loss functions and regularizers.

Abstract

Mathematical modelling, particularly through approaches such as structured sparse support vector machines (SS-SVM), plays a crucial role in processing data with complex feature structures, yet efficient algorithms for distributed large-scale data remain lacking. To address this gap, this paper proposes a unified optimization framework based on a consensus structure. This framework is not only applicable to various loss functions and combined regularization terms but can also be effectively extended to non-convex regularizers, demonstrating strong scalability. Building upon this framework, we develop a distributed parallel alternating direction method of multipliers (ADMM) algorithm to efficiently solve SS-SVMs under distributed data storage. To ensure convergence, we incorporate a Gaussian back-substitution technique. Additionally, for completeness, we introduce a family of sparse group Lasso support vector machine (SGL-SVM) and apply it to music information retrieval. Theoretical analysis confirms that the computational complexity of the proposed algorithm is independent of the choice of regularization terms and loss functions, underscoring the universality of the parallel approach. Experiments on both synthetic and real-world music archive datasets validate the reliability, stability, and efficiency of our algorithm.