Computer vision-based food calorie estimation: dataset, method, and experiment

TL;DR

This paper introduces a new food image dataset with volume and mass data, along with a deep learning detection method, to improve the accuracy of calorie estimation from food images.

Contribution

It provides the first comprehensive dataset with volume and mass records for food images and a deep learning approach for complete calorie estimation.

Findings

The dataset contains 2978 annotated images with volume and mass data.

The proposed Faster R-CNN based method effectively estimates food calories.

The dataset enables evaluation of computer vision methods for calorie estimation.

Abstract

Computer vision has been introduced to estimate calories from food images. But current food image data sets don't contain volume and mass records of foods, which leads to an incomplete calorie estimation. In this paper, we present a novel food image data set with volume and mass records of foods, and a deep learning method for food detection, to make a complete calorie estimation. Our data set includes 2978 images, and every image contains corresponding each food's annotation, volume and mass records, as well as a certain calibration reference. To estimate calorie of food in the proposed data set, a deep learning method using Faster R-CNN first is put forward to detect the food. And the experiment results show our method is effective to estimate calories and our data set contains adequate information for calorie estimation. Our data set is the first released food image data set which can be used to evaluate computer vision-based calorie estimation methods.