Culinary Class Wars: Evaluating LLMs using ASH in Cuisine Transfer Task

TL;DR

This paper introduces the ASH benchmark to evaluate Large Language Models' ability to creatively adapt recipes across cultures, revealing strengths and limitations in culinary cultural understanding.

Contribution

It presents a novel benchmark and methodology for assessing LLMs' culinary creativity and cultural accuracy in cuisine transfer tasks.

Findings

LLMs show varying levels of cultural accuracy in recipe adaptation.

The ASH benchmark effectively evaluates culinary creativity and cultural sensitivity.

Insights into LLMs' strengths and limitations in culinary domain understanding.

Abstract

The advent of Large Language Models (LLMs) have shown promise in various creative domains, including culinary arts. However, many LLMs still struggle to deliver the desired level of culinary creativity, especially when tasked with adapting recipes to meet specific cultural requirements. This study focuses on cuisine transfer-applying elements of one cuisine to another-to assess LLMs' culinary creativity. We employ a diverse set of LLMs to generate and evaluate culturally adapted recipes, comparing their evaluations against LLM and human judgments. We introduce the ASH (authenticity, sensitivity, harmony) benchmark to evaluate LLMs' recipe generation abilities in the cuisine transfer task, assessing their cultural accuracy and creativity in the culinary domain. Our findings reveal crucial insights into both generative and evaluative capabilities of LLMs in the culinary domain, highlighting strengths and limitations in understanding and applying cultural nuances in recipe creation. The code and dataset used in this project will be openly available in \url{http://github.com/dmis-lab/CulinaryASH}.