Open Domain Suggestion Mining: Problem Definition and Datasets

TL;DR

This paper formally defines suggestion mining in open domains, addresses subjective labeling issues, and provides publicly available benchmark datasets across multiple domains.

Contribution

It introduces a formal task definition, annotation procedure, and publicly available datasets for suggestion mining in open domains.

Findings

Proposed a formal definition for suggestion mining

Developed an annotation procedure addressing subjectivity

Provided publicly available labeled datasets

Abstract

We propose a formal definition for the task of suggestion mining in the context of a wide range of open domain applications. Human perception of the term \emph{suggestion} is subjective and this effects the preparation of hand labeled datasets for the task of suggestion mining. Existing work either lacks a formal problem definition and annotation procedure, or provides domain and application specific definitions. Moreover, many previously used manually labeled datasets remain proprietary. We first present an annotation study, and based on our observations propose a formal task definition and annotation procedure for creating benchmark datasets for suggestion mining. With this study, we also provide publicly available labeled datasets for suggestion mining in multiple domains.