Online Visual Robot Tracking and Identification using Deep LSTM Networks

TL;DR

This paper introduces a real-time vision-based system using deep LSTM networks for online detection, tracking, and identification of robots with identical appearance, addressing challenges like occlusion and hardware limitations.

Contribution

It presents a novel data-driven approach employing deep LSTM networks for robot tracking and identification, trained on simulated data and fine-tuned on real data.

Findings

Effective tracking during occlusions

Real-time performance on limited hardware

Promising results on synthetic and real datasets

Abstract

Collaborative robots working on a common task are necessary for many applications. One of the challenges for achieving collaboration in a team of robots is mutual tracking and identification. We present a novel pipeline for online visionbased detection, tracking and identification of robots with a known and identical appearance. Our method runs in realtime on the limited hardware of the observer robot. Unlike previous works addressing robot tracking and identification, we use a data-driven approach based on recurrent neural networks to learn relations between sequential inputs and outputs. We formulate the data association problem as multiple classification problems. A deep LSTM network was trained on a simulated dataset and fine-tuned on small set of real data. Experiments on two challenging datasets, one synthetic and one real, which include long-term occlusions, show promising results.