January Food Benchmark (JFB): A Public Benchmark Dataset and Evaluation Suite for Multimodal Food Analysis

TL;DR

The paper introduces the January Food Benchmark, a new dataset and evaluation framework for multimodal food analysis, enabling standardized assessment of AI models in nutritional analysis.

Contribution

It provides a public dataset, a comprehensive benchmarking framework, and baseline results, advancing standardized evaluation in automated nutritional analysis.

Findings

Specialized model achieves an Overall Score of 86.2

12.1-point improvement over general-purpose models

Provides a valuable dataset and evaluation tools for future research

Abstract

Progress in AI for automated nutritional analysis is critically hampered by the lack of standardized evaluation methodologies and high-quality, real-world benchmark datasets. To address this, we introduce three primary contributions. First, we present the January Food Benchmark (JFB), a publicly available collection of 1,000 food images with human-validated annotations. Second, we detail a comprehensive benchmarking framework, including robust metrics and a novel, application-oriented overall score designed to assess model performance holistically. Third, we provide baseline results from both general-purpose Vision-Language Models (VLMs) and our own specialized model, january/food-vision-v1. Our evaluation demonstrates that the specialized model achieves an Overall Score of 86.2, a 12.1-point improvement over the best-performing general-purpose configuration. This work offers the research community a valuable new evaluation dataset and a rigorous framework to guide and benchmark future developments in automated nutritional analysis.