Efficient Urdu Caption Generation using Attention based LSTM

TL;DR

This paper presents an attention-based LSTM model for automatic Urdu image captioning, filling a language gap with a new dataset and achieving high BLEU scores, thus advancing Urdu language processing in vision-language tasks.

Contribution

It introduces the first Urdu caption generation model using attention-based deep learning and creates a new Urdu dataset based on Flickr8k images.

Findings

Achieved BLEU score of 0.83 on Urdu caption dataset

Improved caption quality with advanced CNN architectures

Demonstrated potential for grammar correction in generated captions

Abstract

Recent advancements in deep learning have created many opportunities to solve real-world problems that remained unsolved for more than a decade. Automatic caption generation is a major research field, and the research community has done a lot of work on it in most common languages like English. Urdu is the national language of Pakistan and also much spoken and understood in the sub-continent region of Pakistan-India, and yet no work has been done for Urdu language caption generation. Our research aims to fill this gap by developing an attention-based deep learning model using techniques of sequence modeling specialized for the Urdu language. We have prepared a dataset in the Urdu language by translating a subset of the "Flickr8k" dataset containing 700 'man' images. We evaluate our proposed technique on this dataset and show that it can achieve a BLEU score of 0.83 in the Urdu language. We improve on the previous state-of-the-art by using better CNN architectures and optimization techniques. Furthermore, we provide a discussion on how the generated captions can be made correct grammar-wise.