SPICE: Smart Projection Interface for Cooking Enhancement

TL;DR

SPICE introduces a tangible user interface that projects recipe information onto cooking surfaces, significantly improving efficiency in cooking tasks through interactive, kitchen-based digital representations.

Contribution

This paper presents SPICE, a novel TUI system for kitchen environments that enhances recipe following by integrating tracking, simulation, and vision large language models.

Findings

Participants completed recipes faster with SPICE.

SPICE reduced the number of stops during cooking.

Participants reported negligible change in perceived difficulty.

Abstract

Tangible User Interfaces (TUI) for human--computer interaction (HCI) provide the user with physical representations of digital information with the aim to overcome the limitations of screen-based interfaces. Although many compelling demonstrations of TUIs exist in the literature, there is a lack of research on TUIs intended for daily two-handed tasks and processes, such as cooking. In response to this gap, we propose SPICE (Smart Projection Interface for Cooking Enhancement). SPICE investigates TUIs in a kitchen setting, aiming to transform the recipe following experience from simply text-based to tangibly interactive. SPICE uses a tracking system, an agent-based simulation software, and vision large language models to create and interpret a kitchen environment where recipe information is projected directly onto the cooking surface. We conducted comparative usability and a validation studies of SPICE, with 30 participants. The results show that participants using SPICE completed the recipe with far less stops and in a substantially shorter time. Despite this, participants self-reported negligible change in feelings of difficulty, which is a direction for future research. Overall, the SPICE project demonstrates the potential of using TUIs to improve everyday activities, paving the way for future research in HCI and new computing interfaces.