Using Keypoint Matching and Interactive Self Attention Network to verify Retail POSMs

TL;DR

This paper presents a computer vision approach combining keypoint matching and an interactive self-attention network to verify the presence of POSM components in retail shelf images, improving accuracy and generalization.

Contribution

It introduces a supervised neural network method that significantly enhances POSM verification accuracy over unsupervised keypoint matching and demonstrates its generalization across different POSM materials.

Findings

Supervised method outperforms baseline in POSM verification accuracy.

The approach generalizes to different POSM materials.

Model trained on private dataset achieves high accuracy.

Abstract

Point of Sale Materials(POSM) are the merchandising and decoration items that are used by companies to communicate product information and offers in retail stores. POSMs are part of companies' retail marketing strategy and are often applied as stylized window displays around retail shelves. In this work, we apply computer vision techniques to the task of verification of POSMs in supermarkets by telling if all desired components of window display are present in a shelf image. We use Convolutional Neural Network based unsupervised keypoint matching as a baseline to verify POSM components and propose a supervised Neural Network based method to enhance the accuracy of baseline by a large margin. We also show that the supervised pipeline is not restricted to the POSM material it is trained on and can generalize. We train and evaluate our model on a private dataset composed of retail shelf images.