Weakly-Supervised Object Detection Learning through Human-Robot Interaction

TL;DR

This paper introduces a pipeline for training object detection models on humanoid robots that leverages weak supervision, online learning, and human-robot interaction to adapt efficiently to new environments with minimal human labeling.

Contribution

It proposes a novel, integrated approach combining weakly supervised learning, teacher-learner pipeline, and online adaptation for effective object detection in humanoid robots.

Findings

Successful real-time deployment on R1 humanoid robot

Reduced human labeling effort through weak supervision

Effective online model re-training in dynamic environments

Abstract

Reliable perception and efficient adaptation to novel conditions are priority skills for humanoids that function in dynamic environments. The vast advancements in latest computer vision research, brought by deep learning methods, are appealing for the robotics community. However, their adoption in applied domains is not straightforward since adapting them to new tasks is strongly demanding in terms of annotated data and optimization time. Nevertheless, robotic platforms, and especially humanoids, present opportunities (such as additional sensors and the chance to explore the environment) that can be exploited to overcome these issues. In this paper, we present a pipeline for efficiently training an object detection system on a humanoid robot. The proposed system allows to iteratively adapt an object detection model to novel scenarios, by exploiting: (i) a teacher-learner pipeline, (ii) weakly supervised learning techniques to reduce the human labeling effort and (iii) an on-line learning approach for fast model re-training. We use the R1 humanoid robot for both testing the proposed pipeline in a real-time application and acquire sequences of images to benchmark the method. We made the code of the application publicly available.