Image Captioning at Will: A Versatile Scheme for Effectively Injecting Sentiments into Image Descriptions
Quanzeng You, Hailin Jin, Jiebo Luo

TL;DR
This paper introduces two simple yet effective models for injecting sentiments into image captions, enabling the generation of emotional descriptions that are more diverse and appealing, outperforming existing methods.
Contribution
The paper presents novel sentiment injection schemes into image captioning models, enhancing emotional expressiveness without compromising semantic accuracy.
Findings
Models outperform state-of-the-art in sentiment captioning
Easy manipulation of generated sentiments for test images
Simpler models achieve better results than previous approaches
Abstract
Automatic image captioning has recently approached human-level performance due to the latest advances in computer vision and natural language understanding. However, most of the current models can only generate plain factual descriptions about the content of a given image. However, for human beings, image caption writing is quite flexible and diverse, where additional language dimensions, such as emotion, humor and language styles, are often incorporated to produce diverse, emotional, or appealing captions. In particular, we are interested in generating sentiment-conveying image descriptions, which has received little attention. The main challenge is how to effectively inject sentiments into the generated captions without altering the semantic matching between the visual content and the generated descriptions. In this work, we propose two different models, which employ different schemes…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Video Analysis and Summarization · Human Pose and Action Recognition
