Specificity measures and reference

TL;DR

This paper empirically evaluates fuzzy measures of referential success for referring expressions with gradual properties, demonstrating their effectiveness in predicting human reference resolution accuracy.

Contribution

It introduces and tests fuzzy measures of success for referring expressions involving gradual properties, showing their predictive validity for human reference resolution.

Findings

Fuzzy success measures predict human accuracy in reference resolution

Certain measures are suitable for evaluating generated referring expressions

Effectiveness shown especially when domain properties are not crisply defined

Abstract

In this paper we study empirically the validity of measures of referential success for referring expressions involving gradual properties. More specifically, we study the ability of several measures of referential success to predict the success of a user in choosing the right object, given a referring expression. Experimental results indicate that certain fuzzy measures of success are able to predict human accuracy in reference resolution. Such measures are therefore suitable for the estimation of the success or otherwise of a referring expression produced by a generation algorithm, especially in case the properties in a domain cannot be assumed to have crisp denotations.