RF-Annotate: Automatic RF-Supervised Image Annotation of Common Objects in Context

TL;DR

RF-Annotate is an autonomous system that uses RFID tags and RGB-D cameras on robots to automatically generate pixel-wise annotations of common objects, facilitating large-scale visual data collection for robotic perception.

Contribution

The paper introduces RF-Annotate, a novel pipeline that leverages RFID tags and robotic motion to automatically annotate images without manual labeling.

Findings

Successfully annotated objects in indoor scenes using RFID and RGB-D data.

Demonstrated the system's effectiveness across various tabletop environments.

Enabled large-scale data collection for training robotic perception models.

Abstract

Wireless tags are increasingly used to track and identify common items of interest such as retail goods, food, medicine, clothing, books, documents, keys, equipment, and more. At the same time, there is a need for labelled visual data featuring such items for the purpose of training object detection and recognition models for robots operating in homes, warehouses, stores, libraries, pharmacies, and so on. In this paper, we ask: can we leverage the tracking and identification capabilities of such tags as a basis for a large-scale automatic image annotation system for robotic perception tasks? We present RF-Annotate, a pipeline for autonomous pixel-wise image annotation which enables robots to collect labelled visual data of objects of interest as they encounter them within their environment. Our pipeline uses unmodified commodity RFID readers and RGB-D cameras, and exploits arbitrary small-scale motions afforded by mobile robotic platforms to spatially map RFIDs to corresponding objects in the scene. Our only assumption is that the objects of interest within the environment are pre-tagged with inexpensive battery-free RFIDs costing 3-15 cents each. We demonstrate the efficacy of our pipeline on several RGB-D sequences of tabletop scenes featuring common objects in a variety of indoor environments.