Semantics or spelling? Probing contextual word embeddings with orthographic noise

TL;DR

This paper investigates whether contextual word embeddings from pretrained language models truly encode semantic information by testing their sensitivity to orthographic noise, revealing they are more affected by input modifications than expected.

Contribution

The study demonstrates that CWEs are highly sensitive to orthographic noise and subword tokenization, challenging their reliability as semantic representations.

Findings

CWEs are sensitive to minimal orthographic changes.

Sensitivity is related to subword tokenization.

CWEs may encode information beyond word meaning.

Abstract

Pretrained language model (PLM) hidden states are frequently employed as contextual word embeddings (CWE): high-dimensional representations that encode semantic information given linguistic context. Across many areas of computational linguistics research, similarity between CWEs is interpreted as semantic similarity. However, it remains unclear exactly what information is encoded in PLM hidden states. We investigate this practice by probing PLM representations using minimal orthographic noise. We expect that if CWEs primarily encode semantic information, a single character swap in the input word will not drastically affect the resulting representation,given sufficient linguistic context. Surprisingly, we find that CWEs generated by popular PLMs are highly sensitive to noise in input data, and that this sensitivity is related to subword tokenization: the fewer tokens used to represent a word at input, the more sensitive its corresponding CWE. This suggests that CWEs capture information unrelated to word-level meaning and can be manipulated through trivial modifications of input data. We conclude that these PLM-derived CWEs may not be reliable semantic proxies, and that caution is warranted when interpreting representational similarity