GATE: A Challenge Set for Gender-Ambiguous Translation Examples
Spencer Rarrick, Ranjita Naik, Varun Mathur, Sundar Poudel, Vishal, Chowdhary

TL;DR
GATE introduces a diverse corpus and evaluation tools for gender-ambiguous translation, addressing biases and improving linguistic coverage in machine translation systems.
Contribution
The paper presents GATE, a new challenging dataset with evaluation tools to enhance gender-ambiguous translation and reduce stereotypical biases.
Findings
GATE enables better evaluation of gender rewriters.
Current systems show limited coverage on ambiguous inputs.
GATE promotes fairer machine translation practices.
Abstract
Although recent years have brought significant progress in improving translation of unambiguously gendered sentences, translation of ambiguously gendered input remains relatively unexplored. When source gender is ambiguous, machine translation models typically default to stereotypical gender roles, perpetuating harmful bias. Recent work has led to the development of "gender rewriters" that generate alternative gender translations on such ambiguous inputs, but such systems are plagued by poor linguistic coverage. To encourage better performance on this task we present and release GATE, a linguistically diverse corpus of gender-ambiguous source sentences along with multiple alternative target language translations. We also provide tools for evaluation and system analysis when using GATE and use them to evaluate our translation rewriter system.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNatural Language Processing Techniques · Text Readability and Simplification · Topic Modeling
