RealAppliance: Let High-fidelity Appliance Assets Controllable and Workable as Aligned Real Manuals

TL;DR

This paper introduces the RealAppliance dataset and benchmark, providing high-fidelity, mechanistically complete appliance assets aligned with manuals to improve simulation-reality transfer in appliance manipulation tasks.

Contribution

It presents a new dataset and benchmark for appliance manipulation, enabling better evaluation and development of models in this domain.

Findings

Model performance insights on manipulation tasks

Identification of gaps in current models

Benchmark for future appliance manipulation research

Abstract

Existing appliance assets suffer from poor rendering, incomplete mechanisms, and misalignment with manuals, leading to simulation-reality gaps that hinder appliance manipulation development. In this work, we introduce the RealAppliance dataset, comprising 100 high-fidelity appliances with complete physical, electronic mechanisms, and program logic aligned with their manuals. Based on these assets, we propose the RealAppliance-Bench benchmark, which evaluates multimodal large language models and embodied manipulation planning models across key tasks in appliance manipulation planning: manual page retrieval, appliance part grounding, open-loop manipulation planning, and closed-loop planning adjustment. Our analysis of model performances on RealAppliance-Bench provides insights for advancing appliance manipulation research