Multi-modal application: Image Memes Generation

TL;DR

This paper presents an end-to-end AI system that generates image memes by selecting appropriate templates based on input sentences and creating matching captions, advancing meme creation automation.

Contribution

It introduces a novel encoder-decoder architecture that combines emotion-based template selection with caption generation for meme creation.

Findings

The system effectively generates relevant memes from input sentences.

The model achieves high accuracy in template selection based on emotional content.

Code and models are publicly available for further research.

Abstract

Meme is an interesting word. Internet memes offer unique insights into the changes in our perception of the world, the media and our own lives. If you surf the Internet for long enough, you will see it somewhere on the Internet. With the rise of social media platforms and convenient image dissemination, Image Meme has gained fame. Image memes have become a kind of pop culture and they play an important role in communication over social media, blogs, and open messages. With the development of artificial intelligence and the widespread use of deep learning, Natural Language Processing (NLP) and Computer Vision (CV) can also be used to solve more problems in life, including meme generation. An Internet meme commonly takes the form of an image and is created by combining a meme template (image) and a caption (natural language sentence). In our project, we propose an end-to-end encoder-decoder architecture meme generator. For a given input sentence, we use the Meme template selection model to determine the emotion it expresses and select the image template. Then generate captions and memes through to the meme caption generator. Code and models are available at github