RobotEQ: Transitioning from Passive Intelligence to Active Intelligence in Embodied AI

TL;DR

RobotEQ introduces a benchmark dataset and evaluation framework to assess robots' understanding of social norms and active intelligence in embodied AI scenarios, highlighting current limitations and potential improvements.

Contribution

This paper presents the first benchmark for active intelligence in embodied AI, including a dataset, evaluation metrics, and analysis of model performance.

Findings

Current models underperform in spatial grounding tasks.

Leveraging external social norm knowledge improves model performance.

Models still lack reliable active social intelligence in embodied scenarios.

Abstract

Embodied AI is a prominent research topic in both academia and industry. Current research centers on completing tasks based on explicit user instructions. However, for robots to integrate into human society, they must understand which actions are permissible and which are prohibited, even without explicit commands. We refer to the user-guided AI as passive intelligence and the unguided AI as active intelligence. This paper introduces RobotEQ, the first benchmark for active intelligence, aiming to assess whether existing models can comprehend and adhere to social norms in embodied scenarios. First, we construct RobotEQ-Data, a dataset consisting of 1,900 egocentric images, spanning 10 representative embodied categories and 56 subcategories. Through extensive manual annotation, we provide 5,353 action judgment questions and 1,286 spatial grounding questions, specifying appropriate robot actions across diverse scenarios. Furthermore, we establish RobotEQ-Bench to evaluate the performance of state-of-the-art models on this task. Experimental results show that current models still fall short in achieving reliable active intelligence, particularly in spatial grounding. Meanwhile, we observe that leveraging RAG techniques to incorporate external social norm knowledge bases can generally enhance performance. This work can facilitate the transition of robotics from user-guided passive manipulation to active social compliance.