Is a Picture Worth Ten Thousand Words in a Review Dataset?

TL;DR

This paper introduces a deep learning framework that tags, captions, and recommends relevant images to enhance review datasets, addressing the challenge of linking images to reviews without explicit mappings.

Contribution

The paper presents a novel deep learning-based approach for automatic image tagging, captioning, and review enhancement in datasets lacking explicit review-image associations.

Findings

The framework achieves high-quality automatic captioning and tagging.

It effectively recommends relevant images to reviews.

Qualitative and quantitative evaluations confirm the framework's effectiveness.

Abstract

While textual reviews have become prominent in many recommendation-based systems, automated frameworks to provide relevant visual cues against text reviews where pictures are not available is a new form of task confronted by data mining and machine learning researchers. Suggestions of pictures that are relevant to the content of a review could significantly benefit the users by increasing the effectiveness of a review. We propose a deep learning-based framework to automatically: (1) tag the images available in a review dataset, (2) generate a caption for each image that does not have one, and (3) enhance each review by recommending relevant images that might not be uploaded by the corresponding reviewer. We evaluate the proposed framework using the Yelp Challenge Dataset. While a subset of the images in this particular dataset are correctly captioned, the majority of the pictures do not have any associated text. Moreover, there is no mapping between reviews and images. Each image has a corresponding business-tag where the picture was taken, though. The overall data setting and unavailability of crucial pieces required for a mapping make the problem of recommending images for reviews a major challenge. Qualitative and quantitative evaluations indicate that our proposed framework provides high quality enhancements through automatic captioning, tagging, and recommendation for mapping reviews and images.