Fluent and Accurate Image Captioning with a Self-Trained Reward Model

TL;DR

This paper introduces Self-Cap, a novel image captioning method that uses a self-trained, contrastive reward model to generate more descriptive and accurate captions, overcoming limitations of traditional and CLIP-based rewards.

Contribution

The paper presents a learnable, contrastive reward model trained with self-generated negatives, improving caption quality and reducing fine-tuning time compared to traditional metrics.

Findings

Enhanced caption descriptiveness and semantic richness.

Reduced fine-tuning time compared to CIDEr-based methods.

Effective on both standard and zero-shot datasets.

Abstract

Fine-tuning image captioning models with hand-crafted rewards like the CIDEr metric has been a classical strategy for promoting caption quality at the sequence level. This approach, however, is known to limit descriptiveness and semantic richness and tends to drive the model towards the style of ground-truth sentences, thus losing detail and specificity. On the contrary, recent attempts to employ image-text models like CLIP as reward have led to grammatically incorrect and repetitive captions. In this paper, we propose Self-Cap, a captioning approach that relies on a learnable reward model based on self-generated negatives that can discriminate captions based on their consistency with the image. Specifically, our discriminator is a fine-tuned contrastive image-text model trained to promote caption correctness while avoiding the aberrations that typically happen when training with a CLIP-based reward. To this end, our discriminator directly incorporates negative samples from a frozen captioner, which significantly improves the quality and richness of the generated captions but also reduces the fine-tuning time in comparison to using the CIDEr score as the sole metric for optimization. Experimental results demonstrate the effectiveness of our training strategy on both standard and zero-shot image captioning datasets.