Robotic Interestingness via Human-Informed Few-Shot Object Detection

TL;DR

This paper presents AirInteraction, a framework enabling robots to recognize interesting objects through few-shot learning with human input, improving adaptability in autonomous exploration while minimizing communication bandwidth.

Contribution

It introduces the first human-informed few-shot object detection framework for autonomous exploration, combining online unsupervised learning with minimal human annotations.

Findings

Effective detection of human-informed objects with few examples

Reduced communication bandwidth through staged learning

Successful evaluation on various interesting scene datasets

Abstract

Interestingness recognition is crucial for decision making in autonomous exploration for mobile robots. Previous methods proposed an unsupervised online learning approach that can adapt to environments and detect interesting scenes quickly, but lack the ability to adapt to human-informed interesting objects. To solve this problem, we introduce a human-interactive framework, AirInteraction, that can detect human-informed objects via few-shot online learning. To reduce the communication bandwidth, we first apply an online unsupervised learning algorithm on the unmanned vehicle for interestingness recognition and then only send the potential interesting scenes to a base-station for human inspection. The human operator is able to draw and provide bounding box annotations for particular interesting objects, which are sent back to the robot to detect similar objects via few-shot learning. Only using few human-labeled examples, the robot can learn novel interesting object categories during the mission and detect interesting scenes that contain the objects. We evaluate our method on various interesting scene recognition datasets. To the best of our knowledge, it is the first human-informed few-shot object detection framework for autonomous exploration.