Neural Fashion Image Captioning : Accounting for Data Diversity

TL;DR

This paper introduces the InFashAIv1 dataset for fashion image captioning, demonstrating that joint training with diverse datasets enhances caption quality for African fashion images, promoting diversity in fashion AI applications.

Contribution

The paper presents a new dataset, InFashAIv1, and shows that combining it with existing datasets improves captioning for diverse fashion styles.

Findings

Joint training improves caption quality for African fashion images.

InFashAIv1 dataset is publicly released to promote diversity.

Transfer learning from Western to African fashion styles is effective.

Abstract

Image captioning has increasingly large domains of application, and fashion is not an exception. Having automatic item descriptions is of great interest for fashion web platforms, sometimes hosting hundreds of thousands of images. This paper is one of the first to tackle image captioning for fashion images. To address dataset diversity issues, we introduced the InFashAIv1 dataset containing almost 16.000 African fashion item images with their titles, prices, and general descriptions. We also used the well-known DeepFashion dataset in addition to InFashAIv1. Captions are generated using the Show and Tell model made of CNN encoder and RNN Decoder. We showed that jointly training the model on both datasets improves captions quality for African style fashion images, suggesting a transfer learning from Western style data. The InFashAIv1 dataset is released on Github to encourage works with more diversity inclusion.