Two-view 3D Reconstruction for Food Volume Estimation

TL;DR

This paper presents a fully automated two-view 3D reconstruction system using mobile images to accurately estimate food volumes, addressing a key challenge in dietary assessment with high speed and low error.

Contribution

It introduces a novel three-stage pipeline for 3D food modeling from two images, enabling accurate volume estimation without user input.

Findings

Achieved less than 10% average error in volume estimation

Processed each dish in approximately 5.5 seconds

System is fully automated and computationally efficient

Abstract

The increasing prevalence of diet-related chronic diseases coupled with the ineffectiveness of traditional diet management methods have resulted in a need for novel tools to accurately and automatically assess meals. Recently, computer vision based systems that use meal images to assess their content have been proposed. Food portion estimation is the most difficult task for individuals assessing their meals and it is also the least studied area. The present paper proposes a three-stage system to calculate portion sizes using two images of a dish acquired by mobile devices. The first stage consists in understanding the configuration of the different views, after which a dense 3D model is built from the two images; finally, this 3D model serves to extract the volume of the different items. The system was extensively tested on 77 real dishes of known volume, and achieved an average error of less than 10% in 5.5 seconds per dish. The proposed pipeline is computationally tractable and requires no user input, making it a viable option for fully automated dietary assessment.