CulinaryCut-VLAP: A Vision-Language-Action-Physics Framework for Food Cutting via a Force-Aware Material Point Method

TL;DR

This paper introduces a physics-based simulation framework and dataset for food cutting, combining vision, language, and force sensing to improve robotic manipulation of deformable materials.

Contribution

It presents a unified VLA dataset and a physically realistic cutting simulator based on MLS-MPM, enabling stable, scalable, and physics-consistent training for deformable object manipulation.

Findings

Developed a force-aware MPM-based cutting simulator.

Created a comprehensive VLA dataset with diverse trajectories and labels.

Demonstrated stable force and energy transfer tracking during cutting.

Abstract

Food cutting is a highly practical yet underexplored application at the intersection of vision and robotic manipulation. The task remains challenging because interactions between the knife and deformable materials are highly nonlinear and often entail large deformations, frequent contact, and topological change, which in turn hinder stable and safe large-scale data collection. To address these challenges, we propose a unified framework that couples a vision-language-action (VLA) dataset with a physically realistic cutting simulator built on the material point method (MPM). Our simulator adopts MLS-MPM as its computational core, reducing numerical dissipation and energy drift while preserving rotational and shear responses even under topology-changing cuts. During cutting, forces and stress distributions are estimated from impulse exchanges between particles and the grid, enabling stable tracking of transient contact forces and energy transfer. We also provide a benchmark dataset that integrates diverse cutting trajectories, multi-view visual observations, and fine-grained language instructions, together with force--torque and tool--pose labels to provide physically consistent training signals. These components realize a learning--evaluation loop that respects the core physics of cutting and establishes a safe, reproducible, and scalable foundation for advancing VLA models in deformable object manipulation.