MetaFood3D: 3D Food Dataset with Nutrition Values
Yuhao Chen, Jiangpeng He, Gautham Vinod, Siddeshwar Raghavan, Chris, Czarnecki, Jinge Ma, Talha Ibn Mahmud, Bruce Coburn, Dayou Mao, Saeejith, Nair, Pengcheng Xi, Alexander Wong, Edward Delp, Fengqing Zhu
TL;DR
MetaFood3D introduces a comprehensive 3D food dataset with detailed nutrition data, aiming to advance food computing and 3D vision tasks by providing diverse, annotated 3D food objects for research and algorithm development.
Contribution
The paper presents MetaFood3D, a new large-scale 3D food dataset with nutrition information, addressing the lack of domain-specific 3D datasets for food computing.
Findings
Enhances food portion estimation algorithms
Highlights the gap between video and 3D data in food analysis
Demonstrates the dataset's utility in synthetic eating occasion generation
Abstract
Food computing is both important and challenging in computer vision (CV). It significantly contributes to the development of CV algorithms due to its frequent presence in datasets across various applications, ranging from classification and instance segmentation to 3D reconstruction. The polymorphic shapes and textures of food, coupled with high variation in forms and vast multimodal information, including language descriptions and nutritional data, make food computing a complex and demanding task for modern CV algorithms. 3D food modeling is a new frontier for addressing food related problems, due to its inherent capability to deal with random camera views and its straightforward representation for calculating food portion size. However, the primary hurdle in the development of algorithms for food object analysis is the lack of nutrition values in existing 3D datasets. Moreover, in the…
Peer Reviews
Decision·ICLR 2025 Conference Withdrawn Submission
- The dataset covers all important aspects of a dataset that can be used for estimating properties like portion size or nutrition content: multi-view images, instance masks, size and nutrition information. - The most important distinction to prior work is the high diversity and large scale while also including 3D information. This is important for nutrition estimation and portion size estimation, as these applications require absolute measurements. - The evaluations of what the dataset can be us
- The paper would be greatly improved if in addition to a dataset, there was a well designed benchmark specifically built around applications like nutrition or portion size estimation, including in the wild testing data collected with various smartphones in various contexts. This would tie it more closely - The application of "food generation" seems a bit arbitrary. It would be very helpful if there was some explanation//justification as to why this is useful and what is the impact of being able
The dataset fills a gap in the field by providing 3D food data with detailed nutritional annotations. The authors benchmark different tasks using the proposed dataset.
- The food categories reflect only American dietary patterns, which may limit the dataset’s applicability for global diets. - The dataset includes only 637 objects, which may restrict the generalizability of models trained on it, especially for categories with high visual variability. - The novelty is limited, which is not enough for ICLR conference. The authors only propose the dataset and benchmarks using existing methods. There is no contribution about proposing methods.
1. It is interesting to establish a 3D dataset for food, which also provides different sources, such as textured meshes, RGB-D videos, and segmentation masks. 2. It provides multiple downstream tasks, including 3D food perception, novel view synthesis, 3D mesh reconstruction, 3D food generation, and food portion estimation.
1. The food list is derived from the American diet, which may limit its applicability and representation of dietary diversity in other regions. 2. Compared to existing datasets, this dataset seems to be not large enough in scale, and its design does not sufficiently highlight the characteristics of the food category.
- A multi-modal dataset on food categories is important in the relevant community. - According to Table 1, the proposed dataset seems to include more types of annotations than any prior datasets. - Presentation is mostly clear. Important stats are provided as well. - A major contribution of this paper is its thorough evaluation of the proposed dataset, with a total number of 4 different tasks.
- My major concern is the scale of the proposed dataset. Although there are 108 different food categories, there are only 637 samples. Compared to OmniObject3D which does not provide nutrition information, the proposed dataset yields only ~20 more categories but with <1/4 number of samples. This limits the contribution and future potential of the dataset --- it can only be used for evaluation not for training. - There is no new method proposed along with the dataset, which weakens the contribut
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSmart Agriculture and AI · Nutritional Studies and Diet · Image Processing and 3D Reconstruction