Switching to Discriminative Image Captioning by Relieving a Bottleneck of Reinforcement Learning

TL;DR

This paper identifies that reinforcement learning in image captioning models limits vocabulary to high-frequency words, reducing discriminativeness, and proposes a simple fine-tuning method to enhance caption detail without losing quality.

Contribution

The paper reveals RL's role in vocabulary limitation and introduces a straightforward fine-tuning approach to improve discriminativeness in image captioning models.

Findings

Enhanced discriminativeness in captions after fine-tuning

Outperformed previous discriminativeness methods with less computation

Maintained overall caption quality while increasing detail

Abstract

Discriminativeness is a desirable feature of image captions: captions should describe the characteristic details of input images. However, recent high-performing captioning models, which are trained with reinforcement learning (RL), tend to generate overly generic captions despite their high performance in various other criteria. First, we investigate the cause of the unexpectedly low discriminativeness and show that RL has a deeply rooted side effect of limiting the output words to high-frequency words. The limited vocabulary is a severe bottleneck for discriminativeness as it is difficult for a model to describe the details beyond its vocabulary. Then, based on this identification of the bottleneck, we drastically recast discriminative image captioning as a much simpler task of encouraging low-frequency word generation. Hinted by long-tail classification and debiasing methods, we propose methods that easily switch off-the-shelf RL models to discriminativeness-aware models with only a single-epoch fine-tuning on the part of the parameters. Extensive experiments demonstrate that our methods significantly enhance the discriminativeness of off-the-shelf RL models and even outperform previous discriminativeness-aware methods with much smaller computational costs. Detailed analysis and human evaluation also verify that our methods boost the discriminativeness without sacrificing the overall quality of captions.