Representing ELMo embeddings as two-dimensional text online
Andrey Kutuzov, Elizaveta Kuzmenko

TL;DR
This paper introduces ELMoViz, a visualization tool integrated into WebVectors that displays contextualized ELMo embeddings as two-dimensional text, aiding understanding of semantic similarities and lexical substitutes in multiple languages.
Contribution
The paper presents a novel visualization module for ELMo embeddings within WebVectors, enabling interactive exploration of contextualized word representations and their linguistic properties.
Findings
Supports multiple languages including Russian, Norwegian, and English.
Allows dynamic selection of ELMo layers for token embeddings.
Provides lexical and corpus information alongside visualizations.
Abstract
We describe a new addition to the WebVectors toolkit which is used to serve word embedding models over the Web. The new ELMoViz module adds support for contextualized embedding architectures, in particular for ELMo models. The provided visualizations follow the metaphor of `two-dimensional text' by showing lexical substitutes: words which are most semantically similar in context to the words of the input sentence. The system allows the user to change the ELMo layers from which token embeddings are inferred. It also conveys corpus information about the query words and their lexical substitutes (namely their frequency tiers and parts of speech). The module is well integrated into the rest of the WebVectors toolkit, providing lexical hyperlinks to word representations in static embedding models. Two web services have already implemented the new functionality with pre-trained ELMo models…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
MethodsTanh Activation · Sigmoid Activation · Long Short-Term Memory · Softmax · Bidirectional LSTM · ELMo
