SemEval-2010 Task 8: Multi-Way Classification of Semantic Relations Between Pairs of Nominals

TL;DR

This paper introduces a new semantic relation classification task for nominals, providing datasets and benchmarking 28 systems from 10 teams to advance computational semantic analysis research.

Contribution

It defines a novel multi-way classification task for semantic relations between nominals and establishes a standard dataset and evaluation framework for future research.

Findings

28 systems participated in the evaluation

10 teams contributed to the results

The task serves as a benchmark for semantic relation classification

Abstract

In response to the continuing research interest in computational semantic analysis, we have proposed a new task for SemEval-2010: multi-way classification of mutually exclusive semantic relations between pairs of nominals. The task is designed to compare different approaches to the problem and to provide a standard testbed for future research. In this paper, we define the task, describe the creation of the datasets, and discuss the results of the participating 28 systems submitted by 10 teams.