Dietary Intake Estimation via Continuous 3D Reconstruction of Food

TL;DR

This paper introduces a novel method for monitoring dietary intake by reconstructing 3D food models from monocular video, enabling precise tracking of food consumption and state changes to improve dietary assessment accuracy.

Contribution

The study presents a new approach using 3D reconstruction from 2D video and pose estimation to monitor food intake, advancing automated dietary monitoring technology.

Findings

Successful 3D reconstruction of food items from monocular video

Effective detection of food state changes during consumption

Potential for improved dietary habit monitoring

Abstract

Monitoring dietary habits is crucial for preventing health risks associated with overeating and undereating, including obesity, diabetes, and cardiovascular diseases. Traditional methods for tracking food intake rely on self-reported data before or after the eating, which are prone to inaccuracies. This study proposes an approach to accurately monitor ingest behaviours by leveraging 3D food models constructed from monocular 2D video. Using COLMAP and pose estimation algorithms, we generate detailed 3D representations of food, allowing us to observe changes in food volume as it is consumed. Experiments with toy models and real food items demonstrate the approach's potential. Meanwhile, we have proposed a new methodology for automated state recognition challenges to accurately detect state changes and maintain model fidelity. The 3D reconstruction approach shows promise in capturing comprehensive dietary behaviour insights, ultimately contributing to the development of automated and accurate dietary monitoring tools.