Size Matters: Reconstructing Real-Scale 3D Models from Monocular Images for Food Portion Estimation

TL;DR

This paper introduces a novel method to accurately recover real-world scale in 3D food models from monocular images, significantly improving volume estimation for dietary assessment and aiding precision nutrition.

Contribution

It presents a new approach that leverages visual features from large datasets to estimate true-to-scale 3D reconstructions from single images, addressing a key challenge in computer vision and digital health.

Findings

Achieves nearly 30% reduction in volume estimation error.

Outperforms existing 3D reconstruction techniques.

Demonstrates effectiveness on two public datasets.

Abstract

The rise of chronic diseases related to diet, such as obesity and diabetes, emphasizes the need for accurate monitoring of food intake. While AI-driven dietary assessment has made strides in recent years, the ill-posed nature of recovering size (portion) information from monocular images for accurate estimation of ``how much did you eat?'' is a pressing challenge. Some 3D reconstruction methods have achieved impressive geometric reconstruction but fail to recover the crucial real-world scale of the reconstructed object, limiting its usage in precision nutrition. In this paper, we bridge the gap between 3D computer vision and digital health by proposing a method that recovers a true-to-scale 3D reconstructed object from a monocular image. Our approach leverages rich visual features extracted from models trained on large-scale datasets to estimate the scale of the reconstructed object. This learned scale enables us to convert single-view 3D reconstructions into true-to-life, physically meaningful models. Extensive experiments and ablation studies on two publicly available datasets show that our method consistently outperforms existing techniques, achieving nearly a 30% reduction in mean absolute volume-estimation error, showcasing its potential to enhance the domain of precision nutrition. Code: https://gitlab.com/viper-purdue/size-matters