MetaFood CVPR 2024 Challenge on Physically Informed 3D Food Reconstruction: Methods and Results

TL;DR

The MetaFood CVPR 2024 challenge advances 3D food reconstruction from 2D images, demonstrating promising results and fostering collaboration to improve dietary monitoring tools.

Contribution

This challenge introduces a benchmark for physically informed 3D food reconstruction from limited images, encouraging innovation and collaboration in the field.

Findings

Promising reconstruction accuracy achieved by participating teams.

Effective methods for reconstructing 3D food models from few images.

Potential for improved portion estimation in dietary assessment.

Abstract

The increasing interest in computer vision applications for nutrition and dietary monitoring has led to the development of advanced 3D reconstruction techniques for food items. However, the scarcity of high-quality data and limited collaboration between industry and academia have constrained progress in this field. Building on recent advancements in 3D reconstruction, we host the MetaFood Workshop and its challenge for Physically Informed 3D Food Reconstruction. This challenge focuses on reconstructing volume-accurate 3D models of food items from 2D images, using a visible checkerboard as a size reference. Participants were tasked with reconstructing 3D models for 20 selected food items of varying difficulty levels: easy, medium, and hard. The easy level provides 200 images, the medium level provides 30 images, and the hard level provides only 1 image for reconstruction. In total, 16 teams submitted results in the final testing phase. The solutions developed in this challenge achieved promising results in 3D food reconstruction, with significant potential for improving portion estimation for dietary assessment and nutritional monitoring. More details about this workshop challenge and access to the dataset can be found at https://sites.google.com/view/cvpr-metafood-2024.