# A Surprisingly Robust Trick for Winograd Schema Challenge

## TL;DR

This paper demonstrates that fine-tuning language models on a related pronoun disambiguation dataset significantly improves performance on the Winograd Schema Challenge, achieving new state-of-the-art results and enhanced robustness.

## Contribution

The authors introduce a simple fine-tuning approach on a related dataset that substantially boosts accuracy on WSC benchmarks, surpassing previous methods.

## Key findings

- Achieved 72.5% on WSC273 and 74.7% on WNLI, surpassing previous state-of-the-art.
- Fine-tuning on a related dataset improves robustness on complex subsets.
- Generated a large unsupervised WSC-like dataset to aid training.

## Abstract

The Winograd Schema Challenge (WSC) dataset WSC273 and its inference counterpart WNLI are popular benchmarks for natural language understanding and commonsense reasoning. In this paper, we show that the performance of three language models on WSC273 strongly improves when fine-tuned on a similar pronoun disambiguation problem dataset (denoted WSCR). We additionally generate a large unsupervised WSC-like dataset. By fine-tuning the BERT language model both on the introduced and on the WSCR dataset, we achieve overall accuracies of 72.5% and 74.7% on WSC273 and WNLI, improving the previous state-of-the-art solutions by 8.8% and 9.6%, respectively. Furthermore, our fine-tuned models are also consistently more robust on the "complex" subsets of WSC273, introduced by Trichelair et al. (2018).