Halo: Learning Semantics-Aware Representations for Cross-Lingual Information Extraction

TL;DR

Halo is a training method that enhances cross-lingual information extraction by making neural models learn semantics-aware representations, improving robustness and generalization without extra parameters.

Contribution

The paper introduces Halo, a novel training approach that enforces local semantic consistency in neural representations for improved cross-lingual extraction.

Findings

Improves robustness of models in low-resource scenarios

Enhances generalization in high-resource settings

Does not increase model complexity or parameters

Abstract

Cross-lingual information extraction (CLIE) is an important and challenging task, especially in low resource scenarios. To tackle this challenge, we propose a training method, called Halo, which enforces the local region of each hidden state of a neural model to only generate target tokens with the same semantic structure tag. This simple but powerful technique enables a neural model to learn semantics-aware representations that are robust to noise, without introducing any extra parameter, thus yielding better generalization in both high and low resource settings.