# MSnet: A BERT-based Network for Gendered Pronoun Resolution

## TL;DR

This paper introduces MSnet, a BERT-based neural network model that improves gendered pronoun resolution by using an attention mechanism and semantic similarity measures, achieving competitive results in a shared task.

## Contribution

The paper presents a novel BERT-based model with an attention mechanism for gendered pronoun resolution, demonstrating state-of-the-art performance and competitive results.

## Key findings

- Reduced multi-class logarithmic loss to 0.3033 in training
- Achieved 0.2795 loss in testing
- Secured 2nd place in stage 2 of the task

## Abstract

The pre-trained BERT model achieves a remarkable state of the art across a wide range of tasks in natural language processing. For solving the gender bias in gendered pronoun resolution task, I propose a novel neural network model based on the pre-trained BERT. This model is a type of mention score classifier and uses an attention mechanism with no parameters to compute the contextual representation of entity span, and a vector to represent the triple-wise semantic similarity among the pronoun and the entities. In stage 1 of the gendered pronoun resolution task, a variant of this model, trained in the fine-tuning approach, reduced the multi-class logarithmic loss to 0.3033 in the 5-fold cross-validation of training set and 0.2795 in testing set. Besides, this variant won the 2nd place with a score at 0.17289 in stage 2 of the task. The code in this paper is available at: https://github.com/ziliwang/MSnet-for-Gendered-PronounResolution