WeVibe: Weight Change Estimation Through Audio-Induced Shelf Vibrations In Autonomous Stores

TL;DR

WeVibe introduces a novel vibration-based system for estimating item weight changes on shelves in autonomous stores, leveraging physics-informed models to achieve accurate, location-independent measurements without specialized sensors.

Contribution

The paper presents the first active vibration sensing system for weight change estimation that models shelf dynamics to enable accurate, location-agnostic measurements in real-store environments.

Findings

Achieved mean absolute error of 38.07g in weight change estimation.

System works effectively with only one sensor and minimal training data.

Accurately differentiates items with weight differences over 100g.

Abstract

Weight change estimation is crucial in various applications, particularly for detecting pick-up and put-back actions when people interact with the shelf while shopping in autonomous stores. Moreover, accurate weight change estimation allows autonomous stores to automatically identify items being picked up or put back, ensuring precise cost estimation. However, the conventional approach of estimating weight changes requires specialized weight-sensing shelves, which are densely deployed weight scales, incurring intensive sensor consumption and high costs. Prior works explored the vibration-based weight sensing method, but they failed when the location of weight change varies. In response to these limitations, we made the following contributions: (1) We propose WeVibe, a first item weight change estimation system through active shelf vibration sensing. The main intuition of the system is that the weight placed on the shelf influences the dynamic vibration response of the shelf, thus altering the shelf vibration patterns. (2) We model a physics-informed relationship between the shelf vibration response and item weight across multiple locations on the shelf based on structural dynamics theory. This relationship is linear and allows easy training of a weight estimation model at a new location without heavy data collection. (3) We evaluate our system on a gondola shelf organized as the real-store settings. WeVibe achieved a mean absolute error down to 38.07g and a standard deviation of 31.2g with one sensor and 10% samples from three weight classes on estimating weight change from 0g to 450g, which can be leveraged for differentiating items with more than 100g differences.