RIO: A Benchmark for Reasoning Intention-Oriented Objects in Open Environments

TL;DR

This paper introduces RIO, a comprehensive dataset designed to advance intention-oriented object detection in open environments by providing diverse, contextually relevant intention descriptions and a large number of images and pairs.

Contribution

The paper presents RIO, a new dataset with natural language intention descriptions and extensive real-world scenarios for reasoning about intention-oriented objects.

Findings

Existing models are evaluated on RIO for intention reasoning.

RIO enables better understanding of intention-object relationships.

The dataset supports diverse and realistic intention detection tasks.

Abstract

Intention-oriented object detection aims to detect desired objects based on specific intentions or requirements. For instance, when we desire to "lie down and rest", we instinctively seek out a suitable option such as a "bed" or a "sofa" that can fulfill our needs. Previous work in this area is limited either by the number of intention descriptions or by the affordance vocabulary available for intention objects. These limitations make it challenging to handle intentions in open environments effectively. To facilitate this research, we construct a comprehensive dataset called Reasoning Intention-Oriented Objects (RIO). In particular, RIO is specifically designed to incorporate diverse real-world scenarios and a wide range of object categories. It offers the following key features: 1) intention descriptions in RIO are represented as natural sentences rather than a mere word or verb phrase, making them more practical and meaningful; 2) the intention descriptions are contextually relevant to the scene, enabling a broader range of potential functionalities associated with the objects; 3) the dataset comprises a total of 40,214 images and 130,585 intention-object pairs. With the proposed RIO, we evaluate the ability of some existing models to reason intention-oriented objects in open environments.