RecipeSnap -- a lightweight image-to-recipe model

TL;DR

RecipeSnap is a lightweight, mobile-friendly image-to-recipe model that significantly reduces computational costs while maintaining competitive accuracy, enabling practical deployment on portable devices.

Contribution

The paper introduces RecipeSnap, a novel lightweight image-to-recipe model using MobileNet-V2, achieving over 90% reduction in memory and computation with state-of-the-art accuracy.

Findings

Reduces memory and computational costs by over 90%.

Achieves 2.0 MedR, comparable to state-of-the-art models.

Suitable for deployment on smartphones.

Abstract

In this paper we want to address the problem of automation for recognition of photographed cooking dishes and generating the corresponding food recipes. Current image-to-recipe models are computation expensive and require powerful GPUs for model training and implementation. High computational cost prevents those existing models from being deployed on portable devices, like smart phones. To solve this issue we introduce a lightweight image-to-recipe prediction model, RecipeSnap, that reduces memory cost and computational cost by more than 90% while still achieving 2.0 MedR, which is in line with the state-of-the-art model. A pre-trained recipe encoder was used to compute recipe embeddings. Recipes from Recipe1M dataset and corresponding recipe embeddings are collected as a recipe library, which are used for image encoder training and image query later. We use MobileNet-V2 as image encoder backbone, which makes our model suitable to portable devices. This model can be further developed into an application for smart phones with a few effort. A comparison of the performance between this lightweight model to other heavy models are presented in this paper. Code, data and models are publicly accessible on github.