SubICap: Towards Subword-informed Image Captioning

TL;DR

This paper introduces SubICap, a novel image captioning approach that models captions at the subword level, enabling better handling of rare and out-of-vocabulary words, and achieves improved performance with a smaller vocabulary.

Contribution

It proposes decomposing words into subwords for image captioning, reducing vocabulary size and enhancing the representation of rare words compared to traditional word-level models.

Findings

Improved metric scores over state-of-the-art models.

Reduced vocabulary size by approximately 90%.

Enhanced handling of rare and out-of-vocabulary words.

Abstract

Existing Image Captioning (IC) systems model words as atomic units in captions and are unable to exploit the structural information in the words. This makes representation of rare words very difficult and out-of-vocabulary words impossible. Moreover, to avoid computational complexity, existing IC models operate over a modest sized vocabulary of frequent words, such that the identity of rare words is lost. In this work we address this common limitation of IC systems in dealing with rare words in the corpora. We decompose words into smaller constituent units 'subwords' and represent captions as a sequence of subwords instead of words. This helps represent all words in the corpora using a significantly lower subword vocabulary, leading to better parameter learning. Using subword language modeling, our captioning system improves various metric scores, with a training vocabulary size approximately 90% less than the baseline and various state-of-the-art word-level models. Our quantitative and qualitative results and analysis signify the efficacy of our proposed approach.