On-Device Emoji Classifier Trained with GPT-based Data Augmentation for a Mobile Keyboard

TL;DR

This paper presents an on-device emoji classifier for mobile keyboards that uses GPT-based data augmentation to improve accuracy, especially for rare emojis, while meeting low memory and latency constraints.

Contribution

It introduces a MobileBert-based emoji classifier augmented with GPT-generated data to handle class imbalance and improve on-device emoji prediction accuracy.

Findings

Enhanced emoji prediction accuracy, especially for rare emojis.

Effective handling of data imbalance through GPT-generated tags.

Improved user engagement with personalized emoji suggestions.

Abstract

Emojis improve communication quality among smart-phone users that use mobile keyboards to exchange text. To predict emojis for users based on input text, we should consider the on-device low memory and time constraints, ensure that the on-device emoji classifier covers a wide range of emoji classes even though the emoji dataset is typically imbalanced, and adapt the emoji classifier output to user favorites. This paper proposes an on-device emoji classifier based on MobileBert with reasonable memory and latency requirements for SwiftKey. To account for the data imbalance, we utilize the widely used GPT to generate one or more tags for each emoji class. For each emoji and corresponding tags, we merge the original set with GPT-generated sentences and label them with this emoji without human intervention to alleviate the data imbalance. At inference time, we interpolate the emoji output with the user history for emojis for better emoji classifications. Results show that the proposed on-device emoji classifier deployed for SwiftKey increases the accuracy performance of emoji prediction particularly on rare emojis and emoji engagement.