Learning to Estimate the Pose of a Peer Robot in a Camera Image by Predicting the States of its LEDs

TL;DR

This paper introduces a method for estimating a peer robot's 6D pose from camera images by predicting its LED states, leveraging cheap unlabeled data to improve localization accuracy.

Contribution

It proposes a pretext task of LED state prediction to enhance 6D pose estimation, reducing the need for extensive labeled data and enabling effective training with minimal ground truth annotations.

Findings

Pretext task improves pose estimation accuracy.

Fine-tuning with few labeled images yields significant gains.

Method generalizes well to unseen environments.

Abstract

We consider the problem of training a fully convolutional network to estimate the relative 6D pose of a robot given a camera image, when the robot is equipped with independent controllable LEDs placed in different parts of its body. The training data is composed by few (or zero) images labeled with a ground truth relative pose and many images labeled only with the true state (\textsc{on} or \textsc{off}) of each of the peer LEDs. The former data is expensive to acquire, requiring external infrastructure for tracking the two robots; the latter is cheap as it can be acquired by two unsupervised robots moving randomly and toggling their LEDs while sharing the true LED states via radio. Training with the latter dataset on estimating the LEDs' state of the peer robot (\emph{pretext task}) promotes learning the relative localization task (\emph{end task}). Experiments on real-world data acquired by two autonomous wheeled robots show that a model trained only on the pretext task successfully learns to localize a peer robot on the image plane; fine-tuning such model on the end task with few labeled images yields statistically significant improvements in 6D relative pose estimation with respect to baselines that do not use pretext-task pre-training, and alternative approaches. Estimating the state of multiple independent LEDs promotes learning to estimate relative heading. The approach works even when a large fraction of training images do not include the peer robot and generalizes well to unseen environments.