What can Neural Referential Form Selectors Learn?

TL;DR

This paper investigates how well neural referential form selectors learn linguistic features influencing referring expressions, revealing that they capture some features effectively while struggling with discourse-level properties.

Contribution

It provides a systematic probing analysis of neural RFS models, highlighting which linguistic features are learned and which remain challenging.

Findings

Features related to referential status and syntactic position are well captured.

Discourse structure properties beyond sentence level are less effectively learned.

Neural RFS models show partial understanding of linguistic features.

Abstract

Despite achieving encouraging results, neural Referring Expression Generation models are often thought to lack transparency. We probed neural Referential Form Selection (RFS) models to find out to what extent the linguistic features influencing the RE form are learnt and captured by state-of-the-art RFS models. The results of 8 probing tasks show that all the defined features were learnt to some extent. The probing tasks pertaining to referential status and syntactic position exhibited the highest performance. The lowest performance was achieved by the probing models designed to predict discourse structure properties beyond the sentence level.