Fast Image Caption Generation with Position Alignment

TL;DR

This paper introduces an improved non-autoregressive image captioning model that uses position alignment and latent variable inference to generate high-quality captions faster than traditional autoregressive methods.

Contribution

It proposes a novel NA captioning framework with position alignment and latent variable inference, achieving faster generation without sacrificing quality.

Findings

Outperforms general NA captioning models in accuracy.

Achieves comparable performance to autoregressive models.

Significantly speeds up caption generation process.

Abstract

Recent neural network models for image captioning usually employ an encoder-decoder architecture, where the decoder adopts a recursive sequence decoding way. However, such autoregressive decoding may result in sequential error accumulation and slow generation which limit the applications in practice. Non-autoregressive (NA) decoding has been proposed to cover these issues but suffers from language quality problem due to the indirect modeling of the target distribution. Towards that end, we propose an improved NA prediction framework to accelerate image captioning. Our decoding part consists of a position alignment to order the words that describe the content detected in the given image, and a fine non-autoregressive decoder to generate elegant descriptions. Furthermore, we introduce an inference strategy that regards position information as a latent variable to guide the further sentence generation. The Experimental results on public datasets show that our proposed model achieves better performance compared to general NA captioning models, while achieves comparable performance as autoregressive image captioning models with a significant speedup.