Deep learning framework for robot for person detection and tracking

TL;DR

This paper presents a real-time deep learning framework for mobile robots to detect, track, and follow a person using stereo vision, combining head detection, a regression tracker, and visual servoing for smooth interaction.

Contribution

It introduces an efficient, real-time tracking system integrating deep learning detection, a regression tracker, and visual servoing, optimized for resource-limited robotic platforms.

Findings

Effective real-time person tracking demonstrated in real environments

Robust head-based detection improves tracking accuracy

Smooth robot movement achieved through visual servoing

Abstract

Robustly tracking a person of interest in the crowd with a robotic platform is one of the cornerstones of human-robot interaction. The robot platform which is limited by the computational power, rapid movements, and occlusions of the target requires an efficient and robust framework to perform tracking. This paper proposes a deep learning framework for tracking a person using a mobile robot with a stereo camera. The proposed system detects a person based on its head, then utilizes the low-cost, high-speed regression network-based tracker to track the person of interest in real-time. The visual servoing of the mobile robot has been designed using a PID controller which utilizes tracker output and depth estimation of the person in subsequent frames, hence providing smooth and adaptive movement of the robot based on target movement. The proposed system has been tested in a real environment, thus proving its effectiveness.