Everything Is All It Takes: A Multipronged Strategy for Zero-Shot Cross-Lingual Information Extraction

TL;DR

This paper proposes a multipronged approach combining data projection and self-training techniques to improve zero-shot cross-lingual information extraction across multiple languages and tasks, outperforming individual strategies.

Contribution

It introduces a comprehensive strategy that integrates multiple techniques and evaluates their combined effectiveness for zero-shot cross-lingual IE across diverse languages and tasks.

Findings

Combining techniques yields better performance than individual methods.

Different techniques perform best for different tasks and languages.

The approach is effective across event extraction, NER, POS tagging, and dependency parsing.

Abstract

Zero-shot cross-lingual information extraction (IE) describes the construction of an IE model for some target language, given existing annotations exclusively in some other language, typically English. While the advance of pretrained multilingual encoders suggests an easy optimism of "train on English, run on any language", we find through a thorough exploration and extension of techniques that a combination of approaches, both new and old, leads to better performance than any one cross-lingual strategy in particular. We explore techniques including data projection and self-training, and how different pretrained encoders impact them. We use English-to-Arabic IE as our initial example, demonstrating strong performance in this setting for event extraction, named entity recognition, part-of-speech tagging, and dependency parsing. We then apply data projection and self-training to three tasks across eight target languages. Because no single set of techniques performs the best across all tasks, we encourage practitioners to explore various configurations of the techniques described in this work when seeking to improve on zero-shot training.