Retrofitting Multilingual Sentence Embeddings with Abstract Meaning Representation

TL;DR

This paper proposes a method to enhance multilingual sentence embeddings by integrating Abstract Meaning Representation (AMR), leading to improved performance across semantic similarity and transfer tasks.

Contribution

It introduces a novel retrofitting approach that incorporates AMR into existing multilingual sentence embeddings, significantly boosting their effectiveness.

Findings

Improved state-of-the-art results on semantic textual similarity.

Enhanced performance on multiple transfer tasks.

Demonstrated robustness across different languages and expressions.

Abstract

We introduce a new method to improve existing multilingual sentence embeddings with Abstract Meaning Representation (AMR). Compared with the original textual input, AMR is a structured semantic representation that presents the core concepts and relations in a sentence explicitly and unambiguously. It also helps reduce surface variations across different expressions and languages. Unlike most prior work that only evaluates the ability to measure semantic similarity, we present a thorough evaluation of existing multilingual sentence embeddings and our improved versions, which include a collection of five transfer tasks in different downstream applications. Experiment results show that retrofitting multilingual sentence embeddings with AMR leads to better state-of-the-art performance on both semantic textual similarity and transfer tasks. Our codebase and evaluation scripts can be found at \url{https://github.com/jcyk/MSE-AMR}.