Grab: Fast and Accurate Sensor Processing for Cashier-Free Shopping

TL;DR

Grab is a practical system that enables cashier-free shopping by accurately tracking customers and their items using existing infrastructure, achieving high precision and reducing infrastructure costs.

Contribution

It introduces a comprehensive approach combining pose tracking, face recognition, arm movement analysis, and sensor fusion to enable cashier-free shopping with minimal store redesign.

Findings

Over 90% precision and recall in real store deployment

System remains effective even with 40% confusing actions

Reduces computing infrastructure investment four-fold

Abstract

Cashier-free shopping systems like Amazon Go improve shopping experience, but can require significant store redesign. In this paper, we propose Grab, a practical system that leverages existing infrastructure and devices to enable cashier-free shopping. Grab needs to accurately identify and track customers, and associate each shopper with items he or she retrieves from shelves. To do this, it uses a keypoint-based pose tracker as a building block for identification and tracking, develops robust feature-based face trackers, and algorithms for associating and tracking arm movements. It also uses a probabilistic framework to fuse readings from camera, weight and RFID sensors in order to accurately assess which shopper picks up which item. In experiments from a pilot deployment in a retail store, Grab can achieve over 90% precision and recall even when 40% of shopping actions are designed to confuse the system. Moreover, Grab has optimizations that help reduce investment in computing infrastructure four-fold.