Generating and Exploiting Large-scale Pseudo Training Data for Zero Pronoun Resolution

TL;DR

This paper introduces a method to automatically generate large-scale pseudo training data for zero pronoun resolution, leveraging a transfer learning approach with a two-step training mechanism, significantly improving performance.

Contribution

It presents a novel approach to generate pseudo data and adapt a reading comprehension model for zero pronoun resolution, reducing reliance on annotated data.

Findings

Achieved 3.1% F-score improvement on OntoNotes 5.0

Successfully transferred a cloze-style model to zero pronoun resolution

Demonstrated effectiveness of pseudo data in improving resolution accuracy

Abstract

Most existing approaches for zero pronoun resolution are heavily relying on annotated data, which is often released by shared task organizers. Therefore, the lack of annotated data becomes a major obstacle in the progress of zero pronoun resolution task. Also, it is expensive to spend manpower on labeling the data for better performance. To alleviate the problem above, in this paper, we propose a simple but novel approach to automatically generate large-scale pseudo training data for zero pronoun resolution. Furthermore, we successfully transfer the cloze-style reading comprehension neural network model into zero pronoun resolution task and propose a two-step training mechanism to overcome the gap between the pseudo training data and the real one. Experimental results show that the proposed approach significantly outperforms the state-of-the-art systems with an absolute improvements of 3.1% F-score on OntoNotes 5.0 data.