Characterizing Departures from Linearity in Word Translation

TL;DR

This paper explores how linear approximations of word translation maps vary across embedding spaces, revealing their non-linear nature and providing insights for improving translation methods.

Contribution

It introduces a method to analyze local linearity in word translation maps, demonstrating their non-linear behavior and its correlation with neighborhood distances.

Findings

Local linear maps vary across embedding spaces

Non-linearity is tightly correlated with neighborhood distance

Results can inform the design of more accurate translation maps

Abstract

We investigate the behavior of maps learned by machine translation methods. The maps translate words by projecting between word embedding spaces of different languages. We locally approximate these maps using linear maps, and find that they vary across the word embedding space. This demonstrates that the underlying maps are non-linear. Importantly, we show that the locally linear maps vary by an amount that is tightly correlated with the distance between the neighborhoods on which they are trained. Our results can be used to test non-linear methods, and to drive the design of more accurate maps for word translation.