Sticker820K: Empowering Interactive Retrieval with Stickers

TL;DR

This paper introduces Sticker820K, a large-scale Chinese sticker dataset with rich annotations, and proposes StickerCLIP, a benchmark model that significantly improves sticker retrieval performance and explores prompt tuning to enhance large language models for sticker retrieval.

Contribution

The paper presents a new large-scale sticker dataset and a specialized benchmark model, StickerCLIP, tailored for sticker retrieval tasks, addressing the limitations of natural image models in emotive sticker data.

Findings

StickerCLIP outperforms CLIP with 66.0% higher mean recall on Sticker820K.

Prompt tuning enables LLMs to perform sticker retrieval through instructions without degrading task quality.

The dataset and models facilitate advanced analysis and retrieval of emotive stickers in communication.

Abstract

Stickers have become a ubiquitous part of modern-day communication, conveying complex emotions through visual imagery. To facilitate the development of more powerful algorithms for analyzing stickers, we propose a large-scale Chinese sticker dataset, namely Sticker820K, which consists of 820k image-text pairs. Each sticker has rich and high-quality textual annotations, including descriptions, optical characters, emotional labels, and style classifications. Although vision-language tasks in the domain of natural images have been well studied, directly applying the those models, such as CLIP, to sticker data is not an optimal solution due to the discrepant nature between natural and emotive image data. Therefore, we propose StickerCLIP as a benchmark model on the Sticker820K dataset. For the text-to-image retrieval task, our StickerCLIP demonstrates strong superiority over the CLIP, which achieves an absolute gain of 66.0\% in mean recall on the Sticker820K test set. Additionally, we endeavor to extend the recently popularized LLM by means of prompt tuning, integrating its ability for sticker retrieval and allowing users to retrieve stickers through instructions. We validate the feasibility of this method, demonstrating the immense potential of prompt tuning in expanding LLM abilities while not affecting the quality of upstream tasks.