VolTex: Food Volume Estimation using Text-Guided Segmentation and Neural Surface Reconstruction

TL;DR

VolTex introduces a text-guided segmentation and neural surface reconstruction framework for precise 3D food volume estimation, enhancing food object selection and reconstruction accuracy in real-world scenes.

Contribution

It presents a novel framework combining text-guided segmentation with neural surface reconstruction for improved food volume estimation.

Findings

Effective segmentation of target food items in complex scenes.

High-fidelity 3D reconstructions enable accurate volume calculations.

Demonstrated superior performance on MetaFood3D dataset.

Abstract

Accurate food volume estimation is crucial for dietary monitoring, medical nutrition management, and food intake analysis. Existing 3D Food Volume estimation methods accurately compute the food volume but lack for food portions selection. We present VolTex, a framework that improves \change{the food object selection} in food volume estimation. Allowing users to specify a target food item via text input to be segmented, our method enables the precise selection of specific food objects in real-world scenes. The segmented object is then reconstructed using the Neural Surface Reconstruction method to generate high-fidelity 3D meshes for volume computation. Extensive evaluations on the MetaFood3D dataset demonstrate the effectiveness of our approach in isolating and reconstructing food items for accurate volume estimation. The source code is accessible at https://github.com/GCVCG/VolTex.