# A Hierarchical Grocery Store Image Dataset with Visual and Semantic Labels

## TL;DR

This paper introduces a comprehensive grocery store image dataset with hierarchical labels and semantic information, aimed at improving assistive technology for visually impaired individuals in shopping scenarios.

## Contribution

It provides a new benchmark dataset with hierarchical and semantic labels, along with baseline results using CNNs and autoencoders for assistive shopping applications.

## Key findings

- Benchmark results for pretrained CNNs on the dataset
- Multi-view variational autoencoder effectively utilizes product information
- Dataset supports development of assistive visual support systems

## Abstract

Image classification models built into visual support systems and other assistive devices need to provide accurate predictions about their environment. We focus on an application of assistive technology for people with visual impairments, for daily activities such as shopping or cooking. In this paper, we provide a new benchmark dataset for a challenging task in this application - classification of fruits, vegetables, and refrigerated products, e.g. milk packages and juice cartons, in grocery stores. To enable the learning process to utilize multiple sources of structured information, this dataset not only contains a large volume of natural images but also includes the corresponding information of the product from an online shopping website. Such information encompasses the hierarchical structure of the object classes, as well as an iconic image of each type of object. This dataset can be used to train and evaluate image classification models for helping visually impaired people in natural environments. Additionally, we provide benchmark results evaluated on pretrained convolutional neural networks often used for image understanding purposes, and also a multi-view variational autoencoder, which is capable of utilizing the rich product information in the dataset.