Combination Strategies for Semantic Role Labeling

TL;DR

This paper evaluates various inference models for semantic role labeling, demonstrating that learning-based strategies, especially those using max-margin classifiers, outperform existing methods in the CoNLL-2005 shared task.

Contribution

It provides the first thorough analysis and comparison of learning-based inference strategies for semantic role labeling, highlighting the advantages of candidate-level combination and max-margin classifiers.

Findings

All proposed inference strategies outperform previous best results.

Combining models at the candidate argument level is most effective.

Learning-based inference with max-margin classifiers yields superior performance.

Abstract

This paper introduces and analyzes a battery of inference models for the problem of semantic role labeling: one based on constraint satisfaction, and several strategies that model the inference as a meta-learning problem using discriminative classifiers. These classifiers are developed with a rich set of novel features that encode proposition and sentence-level information. To our knowledge, this is the first work that: (a) performs a thorough analysis of learning-based inference models for semantic role labeling, and (b) compares several inference strategies in this context. We evaluate the proposed inference strategies in the framework of the CoNLL-2005 shared task using only automatically-generated syntactic information. The extensive experimental evaluation and analysis indicates that all the proposed inference strategies are successful -they all outperform the current best results reported in the CoNLL-2005 evaluation exercise- but each of the proposed approaches has its advantages and disadvantages. Several important traits of a state-of-the-art SRL combination strategy emerge from this analysis: (i) individual models should be combined at the granularity of candidate arguments rather than at the granularity of complete solutions; (ii) the best combination strategy uses an inference model based in learning; and (iii) the learning-based inference benefits from max-margin classifiers and global feedback.