Evaluating Gemini LLM in Food Image-Based Recipe and Nutrition Description with EfficientNet-B4 Visual Backbone

TL;DR

This study evaluates a multimodal food recognition system combining EfficientNet-B4 and Gemini LLM, analyzing trade-offs between accuracy, efficiency, and generative quality using a new Chinese food dataset.

Contribution

It introduces a comprehensive evaluation framework for food recognition systems integrating visual backbones with large language models, including a novel formalization of semantic error propagation.

Findings

EfficientNet-B4 achieves 89.0% Top-1 accuracy, balancing accuracy and efficiency.

Gemini LLM provides high-quality, factual nutritional and recipe generation.

System performance is limited by the visual module's accuracy, especially in semantically similar classes.

Abstract

The proliferation of digital food applications necessitates robust methods for automated nutritional analysis and culinary guidance. This paper presents a comprehensive comparative evaluation of a decoupled, multimodal pipeline for food recognition. We evaluate a system integrating a specialized visual backbone (EfficientNet-B4) with a powerful generative large language model (Google's Gemini LLM). The core objective is to evaluate the trade-offs between visual classification accuracy, model efficiency, and the quality of generative output (nutritional data and recipes). We benchmark this pipeline against alternative vision backbones (VGG-16, ResNet-50, YOLOv8) and a lightweight LLM (Gemma). We introduce a formalization for "Semantic Error Propagation" (SEP) to analyze how classification inaccuracies from the visual module cascade into the generative output. Our analysis is grounded in a new Custom Chinese Food Dataset (CCFD) developed to address cultural bias in public datasets. Experimental results demonstrate that while EfficientNet-B4 (89.0\% Top-1 Acc.) provides the best balance of accuracy and efficiency, and Gemini (9.2/10 Factual Accuracy) provides superior generative quality, the system's overall utility is fundamentally bottlenecked by the visual front-end's perceptive accuracy. We conduct a detailed per-class analysis, identifying high semantic similarity as the most critical failure mode.