Heaviside Low-Rank Support Matrix Machine

TL;DR

This paper introduces HL-SMM, a robust support matrix machine model using Heaviside loss and low-rank constraints, improving classification accuracy and noise robustness for matrix-structured data.

Contribution

The paper proposes a novel HL-SMM model combining Heaviside loss with low-rank constraints, along with theoretical analysis and an efficient optimization algorithm.

Findings

HL-SMM outperforms existing methods in accuracy.

HL-SMM demonstrates enhanced robustness to noise.

Theoretical conditions for optimality are established.

Abstract

Support matrix machine (SMM) is an emerging classification framework that directly handles matrix-structured observations, thereby avoiding the spatial correlations destroyed by vectorization. However, most existing SMM variants rely on convex or nonconvex surrogate loss functions, which may lead to high sensitivity to noise. To address this issue, we propose a novel Heaviside low-rank SMM model called HL-SMM, which leverages the Heaviside loss instead of the common hinge or ramp losses for robustness. Moreover, the low-rank constraint is adopted to accurately characterize the inherent global structure. In theory, we analyze the Karush-Kuhn-Tucker (KKT) points and rigorously prove the sufficient and necessary conditions. In algorithms, we develop an effective proximal alternating minimization (PAM) scheme, where all subproblems have closed-form solutions. Extensive experiments on benchmark datasets validate that the proposed HL-SMM achieves superior classification accuracy and robustness compared to state-of-the-art methods.