EmojiLM: Modeling the New Emoji Language

TL;DR

This paper introduces EmojiLM, a sequence-to-sequence model trained on a large synthesized text-emoji corpus, enabling bidirectional translation and advancing emoji language understanding beyond single emoji prediction.

Contribution

The paper presents a large synthesized text-emoji corpus and a specialized model for bidirectional translation, addressing limitations in current emoji research.

Findings

EmojiLM outperforms strong baselines on benchmarks.

The synthesized corpus improves emoji-related downstream tasks.

Human evaluation confirms model effectiveness.

Abstract

With the rapid development of the internet, online social media welcomes people with different backgrounds through its diverse content. The increasing usage of emoji becomes a noticeable trend thanks to emoji's rich information beyond cultural or linguistic borders. However, the current study on emojis is limited to single emoji prediction and there are limited data resources available for further study of the interesting linguistic phenomenon. To this end, we synthesize a large text-emoji parallel corpus, Text2Emoji, from a large language model. Based on the parallel corpus, we distill a sequence-to-sequence model, EmojiLM, which is specialized in the text-emoji bidirectional translation. Extensive experiments on public benchmarks and human evaluation demonstrate that our proposed model outperforms strong baselines and the parallel corpus benefits emoji-related downstream tasks.