Safe Sample Screening for Robust Support Vector Machine

TL;DR

This paper introduces the first safe sample screening methods for non-convex robust SVMs, significantly reducing computational time while maintaining security guarantees, thus enabling large-scale applications.

Contribution

It proposes two novel safe sample screening rules for non-convex RSVM using CCCP, addressing a gap in existing convex-only screening algorithms.

Findings

Significant reduction in computational time demonstrated on benchmark datasets.

First safe screening methods applicable to non-convex RSVM.

Security guarantees provided for the screening rules.

Abstract

Robust support vector machine (RSVM) has been shown to perform remarkably well to improve the generalization performance of support vector machine under the noisy environment. Unfortunately, in order to handle the non-convexity induced by ramp loss in RSVM, existing RSVM solvers often adopt the DC programming framework which is computationally inefficient for running multiple outer loops. This hinders the application of RSVM to large-scale problems. Safe sample screening that allows for the exclusion of training samples prior to or early in the training process is an effective method to greatly reduce computational time. However, existing safe sample screening algorithms are limited to convex optimization problems while RSVM is a non-convex problem. To address this challenge, in this paper, we propose two safe sample screening rules for RSVM based on the framework of concave-convex procedure (CCCP). Specifically, we provide screening rule for the inner solver of CCCP and another rule for propagating screened samples between two successive solvers of CCCP. To the best of our knowledge, this is the first work of safe sample screening to a non-convex optimization problem. More importantly, we provide the security guarantee to our sample screening rules to RSVM. Experimental results on a variety of benchmark datasets verify that our safe sample screening rules can significantly reduce the computational time.