Unsupervised Pronoun Resolution via Masked Noun-Phrase Prediction

TL;DR

This paper introduces Masked Noun-Phrase Prediction (MNPP), an unsupervised pre-training approach that significantly improves pronoun resolution performance across multiple datasets, outperforming previous methods and baselines.

Contribution

It presents a novel unsupervised pre-training strategy for pronoun resolution that outperforms existing methods and enhances few-shot learning capabilities.

Findings

Outperforms all previous unsupervised methods on pronoun resolution datasets

Achieves higher AUC scores after fine-tuning on WinoGrande splits

Significantly outperforms RoBERTa-large baseline in few-shot settings

Abstract

In this work, we propose Masked Noun-Phrase Prediction (MNPP), a pre-training strategy to tackle pronoun resolution in a fully unsupervised setting. Firstly, We evaluate our pre-trained model on various pronoun resolution datasets without any finetuning. Our method outperforms all previous unsupervised methods on all datasets by large margins. Secondly, we proceed to a few-shot setting where we finetune our pre-trained model on WinoGrande-S and XS separately. Our method outperforms RoBERTa-large baseline with large margins, meanwhile, achieving a higher AUC score after further finetuning on the remaining three official splits of WinoGrande.