Not Your Stereo-Typical Estimator: Combining Vision and Language for Volume Perception

TL;DR

This paper presents a novel multi-modal approach combining stereo vision and natural language to improve object volume estimation from images, outperforming vision-only methods.

Contribution

The authors introduce a new method that fuses stereo visual cues with textual priors for more accurate volume estimation, demonstrating significant improvements over existing techniques.

Findings

Text-guided approach outperforms vision-only baselines on public datasets.

Leveraging textual priors effectively guides the volume estimation process.

The method simplifies the integration of multi-modal data for regression tasks.

Abstract

Accurate volume estimation of objects from visual data is a long-standing challenge in computer vision with significant applications in robotics, logistics, and smart health. Existing methods often rely on complex 3D reconstruction pipelines or struggle with the ambiguity inherent in single-view images. To address these limitations, we introduce a new method that fuses implicit 3D cues from stereo vision with explicit prior knowledge from natural language text. Our approach extracts deep features from a stereo image pair and a descriptive text prompt that contains the object's class and an approximate volume, then integrates them using a simple yet effective projection layer into a unified, multi-modal representation for regression. We conduct extensive experiments on public datasets demonstrating that our text-guided approach significantly outperforms vision-only baselines. Our findings show that leveraging even simple textual priors can effectively guide the volume estimation task, paving the way for more context-aware visual measurement systems. Code: https://gitlab.com/viper-purdue/stereo-typical-estimator.