Chittron: An Automatic Bangla Image Captioning System

TL;DR

This paper presents Chittron, an automatic Bangla image captioning system developed using a new dataset of 16,000 images, combining a pre-trained VGG16 with LSTM layers to generate captions in Bangla.

Contribution

The paper introduces the first Bangla image captioning system and a new dataset, addressing language resource gaps and demonstrating promising results.

Findings

Model generates Bangla captions with reasonable accuracy.

BLEU scores indicate effective language modeling.

Dataset of 16,000 images supports future research.

Abstract

Automatic image caption generation aims to produce an accurate description of an image in natural language automatically. However, Bangla, the fifth most widely spoken language in the world, is lagging considerably in the research and development of such domain. Besides, while there are many established data sets to related to image annotation in English, no such resource exists for Bangla yet. Hence, this paper outlines the development of "Chittron", an automatic image captioning system in Bangla. Moreover, to address the data set availability issue, a collection of 16,000 Bangladeshi contextual images has been accumulated and manually annotated in Bangla. This data set is then used to train a model which integrates a pre-trained VGG16 image embedding model with stacked LSTM layers. The model is trained to predict the caption when the input is an image, one word at a time. The results show that the model has successfully been able to learn a working language model and to generate captions of images quite accurately in many cases. The results are evaluated mainly qualitatively. However, BLEU scores are also reported. It is expected that a better result can be obtained with a bigger and more varied data set.