# Towards Generating Stylized Image Captions via Adversarial Training

## TL;DR

This paper introduces ATTEND-GAN, a novel image captioning model that combines attention mechanisms with adversarial training to generate stylistically diverse and content-accurate captions, outperforming existing methods.

## Contribution

The paper presents a new model that effectively integrates attention and adversarial training to enhance stylistic diversity and content accuracy in image captioning.

## Key findings

- Outperforms state-of-the-art models in style and accuracy
- Generates captions with a wider range of stylistic adjectives
- Produces more human-like variability in captions

## Abstract

While most image captioning aims to generate objective descriptions of images, the last few years have seen work on generating visually grounded image captions which have a specific style (e.g., incorporating positive or negative sentiment). However, because the stylistic component is typically the last part of training, current models usually pay more attention to the style at the expense of accurate content description. In addition, there is a lack of variability in terms of the stylistic aspects. To address these issues, we propose an image captioning model called ATTEND-GAN which has two core components: first, an attention-based caption generator to strongly correlate different parts of an image with different parts of a caption; and second, an adversarial training mechanism to assist the caption generator to add diverse stylistic components to the generated captions. Because of these components, ATTEND-GAN can generate correlated captions as well as more human-like variability of stylistic patterns. Our system outperforms the state-of-the-art as well as a collection of our baseline models. A linguistic analysis of the generated captions demonstrates that captions generated using ATTEND-GAN have a wider range of stylistic adjectives and adjective-noun pairs.