Screening Gender Transfer in Neural Machine Translation

TL;DR

This paper investigates how gender information is transferred within neural machine translation systems, revealing multiple pathways for gender transfer and analyzing the flow of gender-related information in encoder-decoder architectures.

Contribution

It introduces methods to analyze gender transfer in NMT, demonstrating that gender information exists in all token representations and follows multiple pathways.

Findings

Gender information is present in all token representations.

Multiple pathways facilitate gender transfer in NMT.

Gender transfer can be influenced by internal representations.

Abstract

This paper aims at identifying the information flow in state-of-the-art machine translation systems, taking as example the transfer of gender when translating from French into English. Using a controlled set of examples, we experiment several ways to investigate how gender information circulates in a encoder-decoder architecture considering both probing techniques as well as interventions on the internal representations used in the MT system. Our results show that gender information can be found in all token representations built by the encoder and the decoder and lead us to conclude that there are multiple pathways for gender transfer.