VSD2M: A Large-scale Vision-language Sticker Dataset for Multi-frame Animated Sticker Generation

TL;DR

This paper introduces VSD2M, the largest dataset for animated sticker generation, and proposes a new Spatial Temporal Interaction layer to enhance video generation methods for creating animated stickers from text prompts.

Contribution

The paper constructs the first large-scale vision-language sticker dataset VSD2M and develops a novel STI layer to improve animated sticker generation performance.

Findings

VSD2M contains two million static and animated stickers.

The STI layer improves information utilization in video generation.

Baseline models trained on VSD2M show promising results.

Abstract

As a common form of communication in social media,stickers win users' love in the internet scenarios, for their ability to convey emotions in a vivid, cute, and interesting way. People prefer to get an appropriate sticker through retrieval rather than creation for the reason that creating a sticker is time-consuming and relies on rule-based creative tools with limited capabilities. Nowadays, advanced text-to-video algorithms have spawned numerous general video generation systems that allow users to customize high-quality, photo-realistic videos by only providing simple text prompts. However, creating customized animated stickers, which have lower frame rates and more abstract semantics than videos, is greatly hindered by difficulties in data acquisition and incomplete benchmarks. To facilitate the exploration of researchers in animated sticker generation (ASG) field, we firstly construct the currently largest vision-language sticker dataset named VSD2M at a two-million scale that contains static and animated stickers. Secondly, to improve the performance of traditional video generation methods on ASG tasks with discrete characteristics, we propose a Spatial Temporal Interaction (STI) layer that utilizes semantic interaction and detail preservation to address the issue of insufficient information utilization. Moreover, we train baselines with several video generation methods (e.g., transformer-based, diffusion-based methods) on VSD2M and conduct a detailed analysis to establish systemic supervision on ASG task. To the best of our knowledge, this is the most comprehensive large-scale benchmark for multi-frame animated sticker generation, and we hope this work can provide valuable inspiration for other scholars in intelligent creation.