Machine Learning approaches to do size based reasoning on Retail Shelf objects to classify product variants

TL;DR

This paper presents machine learning methods to determine product size variants on retail shelves by analyzing object detection and classification outputs, addressing challenges posed by similar-looking variants and irregular stacking.

Contribution

It introduces a novel approach combining object detection, classification, and size-based reasoning, including neural network methods for irregularly stacked products.

Findings

Gradient boosting performs well on clear product facings.

Neural network method effectively handles irregular stacking.

Proposed methods improve size variant classification accuracy.

Abstract

There has been a surge in the number of Machine Learning methods to analyze products kept on retail shelves images. Deep learning based computer vision methods can be used to detect products on retail shelves and then classify them. However, there are different sized variants of products which look exactly the same visually and the method to differentiate them is to look at their relative sizes with other products on shelves. This makes the process of deciphering the sized based variants from each other using computer vision algorithms alone impractical. In this work, we propose methods to ascertain the size variant of the product as a downstream task to an object detector which extracts products from shelf and a classifier which determines product brand. Product variant determination is the task which assigns a product variant to products of a brand based on the size of bounding boxes and brands predicted by classifier. While gradient boosting based methods work well for products whose facings are clear and distinct, a noise accommodating Neural Network method is proposed for cases where the products are stacked irregularly.