A Dataset for Developing and Benchmarking Active Vision

TL;DR

This paper introduces a comprehensive dataset of RGB-D images and bounding boxes for indoor robotic vision, enabling benchmarking and development of active vision and object detection systems.

Contribution

It provides a new large-scale dataset for indoor robotic vision tasks and demonstrates its utility for training object detectors and active vision systems.

Findings

State-of-the-art object detectors are affected by scale, occlusion, and viewing angle.

The dataset enables effective training of deep networks for next best move prediction.

Active vision systems can be improved using the provided dataset.

Abstract

We present a new public dataset with a focus on simulating robotic vision tasks in everyday indoor environments using real imagery. The dataset includes 20,000+ RGB-D images and 50,000+ 2D bounding boxes of object instances densely captured in 9 unique scenes. We train a fast object category detector for instance detection on our data. Using the dataset we show that, although increasingly accurate and fast, the state of the art for object detection is still severely impacted by object scale, occlusion, and viewing direction all of which matter for robotics applications. We next validate the dataset for simulating active vision, and use the dataset to develop and evaluate a deep-network-based system for next best move prediction for object classification using reinforcement learning. Our dataset is available for download at cs.unc.edu/~ammirato/active_vision_dataset_website/.