Local Support Vector Machines:Formulation and Analysis

TL;DR

This paper introduces a generalized formulation of Local Support Vector Machines, analyzes their consistency and convergence properties, and provides generalization error bounds, connecting them to local polynomial methods in nonparametric estimation.

Contribution

It formulates a generalized version of LSVMs, establishes their Bayes consistency, convergence rates, and generalization bounds, linking them to local polynomial learning.

Findings

LSVMs are Bayes consistent under certain conditions

Convergence rates for local risk are established

Generalization error bounds are derived using stability arguments

Abstract

We provide a formulation for Local Support Vector Machines (LSVMs) that generalizes previous formulations, and brings out the explicit connections to local polynomial learning used in nonparametric estimation literature. We investigate the simplest type of LSVMs called Local Linear Support Vector Machines (LLSVMs). For the first time we establish conditions under which LLSVMs make Bayes consistent predictions at each test point $x_0$. We also establish rates at which the local risk of LLSVMs converges to the minimum value of expected local risk at each point $x_0$. Using stability arguments we establish generalization error bounds for LLSVMs.