The Structured Weighted Violations Perceptron Algorithm

TL;DR

The paper introduces the Structured Weighted Violations Perceptron (SWVP), a novel structured prediction algorithm that leverages internal label structures, with proven convergence and superior performance over CSP in synthetic and initial parsing tasks.

Contribution

SWVP generalizes the Collins Structured Perceptron by explicitly exploiting label structure, offering tighter bounds and improved empirical results.

Findings

SWVP outperforms CSP in synthetic HMM data experiments.

SWVP provides tighter mistake and generalization bounds than CSP.

Initial dependency parsing results are promising.

Abstract

We present the Structured Weighted Violations Perceptron (SWVP) algorithm, a new structured prediction algorithm that generalizes the Collins Structured Perceptron (CSP). Unlike CSP, the update rule of SWVP explicitly exploits the internal structure of the predicted labels. We prove the convergence of SWVP for linearly separable training sets, provide mistake and generalization bounds, and show that in the general case these bounds are tighter than those of the CSP special case. In synthetic data experiments with data drawn from an HMM, various variants of SWVP substantially outperform its CSP special case. SWVP also provides encouraging initial dependency parsing results.