TextMage: The Automated Bangla Caption Generator Based On Deep Learning

TL;DR

TextMage is a deep learning-based system designed to generate Bengali captions for images within the Bangladeshi geographical context, addressing language and regional specificity issues in image captioning.

Contribution

It introduces a Bengali image captioning model trained on a new dataset, BanglaLekhaImageCaptions, tailored for the Bangladeshi context, enhancing regional language understanding.

Findings

Model achieved promising captioning accuracy.

System effectively captures regional linguistic nuances.

Demonstrates feasibility of Bengali image captioning in specific locales.

Abstract

Neural Networks and Deep Learning have seen an upsurge of research in the past decade due to the improved results. Generates text from the given image is a crucial task that requires the combination of both sectors which are computer vision and natural language processing in order to understand an image and represent it using a natural language. However existing works have all been done on a particular lingual domain and on the same set of data. This leads to the systems being developed to perform poorly on images that belong to specific locales' geographical context. TextMage is a system that is capable of understanding visual scenes that belong to the Bangladeshi geographical context and use its knowledge to represent what it understands in Bengali. Hence, we have trained a model on our previously developed and published dataset named BanglaLekhaImageCaptions. This dataset contains 9,154 images along with two annotations for each image. In order to access performance, the proposed model has been implemented and evaluated.