Learning to See Inside Opaque Liquid Containers using Speckle Vibrometry

TL;DR

This paper introduces a novel speckle vibrometry method combined with transformer analysis to remotely determine the liquid fill levels inside opaque containers by sensing surface vibrations, expanding computer vision capabilities beyond surface inspection.

Contribution

It presents the first speckle-based vibration sensing system and a transformer model for non-invasively estimating liquid levels in sealed opaque containers, generalizing to unseen instances.

Findings

Accurately estimates liquid levels across various containers.

System is invariant to vibration source and generalizes to unseen instances.

Enables remote inspection without physical contact or manual weighing.

Abstract

Computer vision seeks to infer a wide range of information about objects and events. However, vision systems based on conventional imaging are limited to extracting information only from the visible surfaces of scene objects. For instance, a vision system can detect and identify a Coke can in the scene, but it cannot determine whether the can is full or empty. In this paper, we aim to expand the scope of computer vision to include the novel task of inferring the hidden liquid levels of opaque containers by sensing the tiny vibrations on their surfaces. Our method provides a first-of-a-kind way to inspect the fill level of multiple sealed containers remotely, at once, without needing physical manipulation and manual weighing. First, we propose a novel speckle-based vibration sensing system for simultaneously capturing scene vibrations on a 2D grid of points. We use our system to efficiently and remotely capture a dataset of vibration responses for a variety of everyday liquid containers. Then, we develop a transformer-based approach for analyzing the captured vibrations and classifying the container type and its hidden liquid level at the time of measurement. Our architecture is invariant to the vibration source, yielding correct liquid level estimates for controlled and ambient scene sound sources. Moreover, our model generalizes to unseen container instances within known classes (e.g., training on five Coke cans of a six-pack, testing on a sixth) and fluid levels. We demonstrate our method by recovering liquid levels from various everyday containers.