Towards the Creation of a Nutrition and Food Group Based Image Database

TL;DR

This paper introduces a new food image database linked with detailed food group and nutrient information, aiming to improve fine-grained food classification for dietary assessment.

Contribution

It presents a framework and protocol for creating a nutrition and food group based image database with hierarchical categorization and nutrient linkage.

Findings

Built a database with 16,114 images across 74 food sub-categories.

Linked food images to 1,865 USDA food codes and nutrient data.

Developed a web-based annotation tool for efficient database creation.

Abstract

Food classification is critical to the analysis of nutrients comprising foods reported in dietary assessment. Advances in mobile and wearable sensors, combined with new image based methods, particularly deep learning based approaches, have shown great promise to improve the accuracy of food classification to assess dietary intake. However, these approaches are data-hungry and their performances are heavily reliant on the quantity and quality of the available datasets for training the food classification model. Existing food image datasets are not suitable for fine-grained food classification and the following nutrition analysis as they lack fine-grained and transparently derived food group based identification which are often provided by trained dietitians with expert domain knowledge. In this paper, we propose a framework to create a nutrition and food group based image database that contains both visual and hierarchical food categorization information to enhance links to the nutrient profile of each food. We design a protocol for linking food group based food codes in the U.S. Department of Agriculture's (USDA) Food and Nutrient Database for Dietary Studies (FNDDS) to a food image dataset, and implement a web-based annotation tool for efficient deployment of this protocol.Our proposed method is used to build a nutrition and food group based image database including 16,114 food images representing the 74 most frequently consumed What We Eat in America (WWEIA) food sub-categories in the United States with 1,865 USDA food code matched to a nutrient database, the USDA FNDDS nutrient database.