Deep Cooking: Predicting Relative Food Ingredient Amounts from Images

TL;DR

This paper introduces deep learning models that predict both ingredients and their relative amounts from food images, utilizing semi-automatic data integration, with promising results on internet recipe datasets.

Contribution

It presents the first models capable of estimating ingredient quantities from images, combining sparse and dense predictions with innovative data pre-processing.

Findings

Models achieve encouraging accuracy in ingredient and amount prediction.

Effective semi-automatic data integration improves model performance.

Demonstrates feasibility of quantitative food image analysis.

Abstract

In this paper, we study the novel problem of not only predicting ingredients from a food image, but also predicting the relative amounts of the detected ingredients. We propose two prediction-based models using deep learning that output sparse and dense predictions, coupled with important semi-automatic multi-database integrative data pre-processing, to solve the problem. Experiments on a dataset of recipes collected from the Internet show the models generate encouraging experimental results.