NutritionVerse-Synth: An Open Access Synthetically Generated 2D Food Scene Dataset for Dietary Intake Estimation

TL;DR

NutritionVerse-Synth introduces a large-scale, photorealistic synthetic food image dataset with comprehensive annotations, aiming to enhance automated dietary intake estimation through diverse, controllable data for computer vision models.

Contribution

The paper presents NV-Synth, the largest open-source synthetic food dataset with detailed annotations, generated via physics-based simulations to improve dietary monitoring models.

Findings

NV-Synth contains 84,984 images from 7,082 3D scenes.

The dataset offers diverse food types, viewpoints, and lighting conditions.

Source code for data generation is publicly available.

Abstract

Manually tracking nutritional intake via food diaries is error-prone and burdensome. Automated computer vision techniques show promise for dietary monitoring but require large and diverse food image datasets. To address this need, we introduce NutritionVerse-Synth (NV-Synth), a large-scale synthetic food image dataset. NV-Synth contains 84,984 photorealistic meal images rendered from 7,082 dynamically plated 3D scenes. Each scene is captured from 12 viewpoints and includes perfect ground truth annotations such as RGB, depth, semantic, instance, and amodal segmentation masks, bounding boxes, and detailed nutritional information per food item. We demonstrate the diversity of NV-Synth across foods, compositions, viewpoints, and lighting. As the largest open-source synthetic food dataset, NV-Synth highlights the value of physics-based simulations for enabling scalable and controllable generation of diverse photorealistic meal images to overcome data limitations and drive advancements in automated dietary assessment using computer vision. In addition to the dataset, the source code for our data generation framework is also made publicly available at https://saeejithnair.github.io/nvsynth.