MyFood: A Food Segmentation and Classification System to Aid Nutritional Monitoring

TL;DR

MyFood is an intelligent food recognition system that segments and classifies food images to assist users in nutritional monitoring, outperforming existing solutions and utilizing a new dataset of Brazilian foods.

Contribution

This work introduces a novel system integrating food segmentation and classification with a mobile app, and provides a comparative analysis of state-of-the-art algorithms on a new Brazilian food dataset.

Findings

The system achieved superior accuracy compared to existing market solutions.

DeepLabV3+ and Mask R-CNN performed best among tested algorithms.

The publicly available dataset facilitates future research in food recognition.

Abstract

The absence of food monitoring has contributed significantly to the increase in the population's weight. Due to the lack of time and busy routines, most people do not control and record what is consumed in their diet. Some solutions have been proposed in computer vision to recognize food images, but few are specialized in nutritional monitoring. This work presents the development of an intelligent system that classifies and segments food presented in images to help the automatic monitoring of user diet and nutritional intake. This work shows a comparative study of state-of-the-art methods for image classification and segmentation, applied to food recognition. In our methodology, we compare the FCN, ENet, SegNet, DeepLabV3+, and Mask RCNN algorithms. We build a dataset composed of the most consumed Brazilian food types, containing nine classes and a total of 1250 images. The models were evaluated using the following metrics: Intersection over Union, Sensitivity, Specificity, Balanced Precision, and Positive Predefined Value. We also propose an system integrated into a mobile application that automatically recognizes and estimates the nutrients in a meal, assisting people with better nutritional monitoring. The proposed solution showed better results than the existing ones in the market. The dataset is publicly available at the following link http://doi.org/10.5281/zenodo.4041488